Protocol rules: Difference between revisions

From Bitcoin Wiki
Jump to navigation Jump to search
Sgornick (talk | contribs)
Add link to Clients category in opening sentence.
Furunodo (talk | contribs)
 
(36 intermediate revisions by 19 users not shown)
Line 1: Line 1:
'''Rules''' for [[:Cateogry:clients]].
'''Rules''' for [[:Category:Clients|clients]].


The wiki substantially documents the Bitcoin protocol, but equally important are the rules used by the client to process messages. It's crucial that clients follow certain rules in order to maintain consistency across the network, and to protect the Bitcoin security guarantees.
The wiki substantially documents the [[Protocol_specification|Bitcoin protocol]], but equally important are the rules used by the client to process messages. It's crucial that clients follow certain rules in order to maintain consistency across the network, and to protect the Bitcoin security guarantees.


Here, the focus is on handling tx and block messages, because that is the tricky logic. This will skip over the method of requesting and forwarding these messages for now, and describe what to do when they are received. Also, this will describe the minimal data structures in rather abstract terms, ignoring the client's various indexes, maps and hash tables used for efficiency. This will be a conceptual description. This is all based on a fairly literal reading of the source code.
Here, the focus is on handling tx and block messages, because that is the tricky logic. This will skip over the method of requesting and forwarding these messages for now, and describe what to do when they are received. Also, this will describe the minimal data structures in rather abstract terms, ignoring the client's various indexes, maps and hash tables used for efficiency. This will be a conceptual description. This is all based on a fairly literal reading of the source code.
Line 9: Line 9:
== Data structures ==
== Data structures ==


The main data structures are ''transactions'' and ''blocks''. Blocks are composed of the ''block header'' followed by transactions in the block. Transactions are identified by their hash; blocks by the hash of their header. Blocks have prev pointers that link them into a graph.
The main data structures are [[transaction]]s and [[block]]s. Blocks are composed of the ''block header'' followed by transactions in the block. Transactions are identified by their hash; blocks by the hash of their header. Blocks have prev pointers that link them into a graph.


Conceptually, the client has the following data structures:
Conceptually, the client has the following data structures:
Line 36: Line 36:
: these are blocks which don't link into the main branch, normally because of a missing predecessor or nth-level predecessor
: these are blocks which don't link into the main branch, normally because of a missing predecessor or nth-level predecessor


Blocks in the first two categories form a tree rooted at the genesis block, linked by the prev pointer, which points toward the root. (It is a very linear tree with few and short branches off the main branch.) The main branch is defined as the branch with highest total difficulty, summing the difficulties for each block in the branch.
Blocks in the first two categories form a tree rooted at the [[genesis block]], linked by the prev pointer, which points toward the root. (It is a very linear tree with few and short branches off the main branch.) The main branch is defined as the branch with highest total difficulty, summing the difficulties for each block in the branch.


See also [[Block Status]].


== "tx" messages ==
== Difficulty change ==


These messages hold a single transaction.
The difficulty changes every 2016 blocks.  This choice is designed to occur approximately every two weeks.
 
: 2 weeks / 10 minutes = 14 * 24 * 60 / 10 = 2016
 
Once 2016 blocks has been reached we loop back until we hit the 2016th block before the current one. We find the difference in time between the current block and that one. This difference (called the actual timespan) is limited in bounds between [2 weeks/4, 2 weeks*4].
 
Then we get the last target for this old 2 week window and multiply it by the ratio of the actual timespan / the target timespan (2 weeks in secs).
 
: new target = old target * time for 2016 blocks / 2 weeks.
 
If the old set of blocks completed too fast then the target is lowered (difficulty goes up) ensuring it takes longer to solve these new blocks... and vice versa. This way the difficulty oscillates around the ideal of 2 weeks (and 10 mins per block).
 
== Block creation fee  ==
 
The block creation fee changes at every 210000 blocks.
The block creation fee is a function of block height on the chain (genesis=0), and is calculated using 64 bit integer operations
(in satoshis) as:
 
:(50 * 100000000) >> (height / 210000)
 
The block creation fee started with 50 BTC, has fallen to 25 BTC at block 210000, fell to 12.5 BTC at block 420000, to 6.25 at block 630000, will halve every 210000 blocks (roughly 4 years) and finally go down to 0 satoshi with block 6930000 around the year 2140.
The block creation fee of all coinbase transactions will sum up to  2099999997690000 satoshis, practically 21million BTC.
 
== [[Protocol_specification#tx|"tx"]] messages ==
 
These messages hold a single [[transaction]].


# Check syntactic correctness
# Check syntactic correctness
# Make sure neither in or out lists are empty
# Make sure neither in or out lists are empty
# Size in bytes < MAX_BLOCK_SIZE
# Size in bytes <= MAX_BLOCK_SIZE
# Each output value, as well as the total, must be in legal money range
# Each output value, as well as the total, must be in legal money range
# Make sure none of the inputs have hash=0, n=-1 (''coinbase'' transactions)
# Make sure none of the inputs have hash=0, n=-1 (''coinbase'' transactions)
# Check that nLockTime <= INT_MAX, size in bytes >= 100, and sig opcount <= 2
# Check that nLockTime <= INT_MAX<ref>nLockTime must not exceed 31 bits, as some clients will interpret it incorrectly</ref>, size in bytes >= 100<ref>A valid transaction requires at least 100 bytes. If it's any less, the transaction is not valid</ref>, and sig opcount <= 2<ref>The number of signature operands in the signature (no, that is not redundant) for standard transactions will never exceed two</ref>
# Reject "nonstandard" transactions: scriptSig doing anything other than pushing numbers on the stack, or scriptPubkey not matching the two usual forms
# Reject "nonstandard" transactions: scriptSig doing anything other than pushing numbers on the stack, or scriptPubkey not matching the two usual forms<ref>Note that this is not a hard requirement on clients.</ref>
# Reject if we already have matching tx in the pool, or in a block in the main branch
# Reject if we already have matching tx in the pool, or in a block in the main branch
# Reject if any other tx in the pool uses the same transaction output as one used by this tx.
# For each input, if the referenced output exists in any other tx in the pool, reject this transaction.<ref>Note that this is not a hard requirement on clients.  The network-enforced rule is that only <i>one</i> transaction spending a particular output can be in the blockchain, thus preventing double-spending.  Technically miners can choose which one they want to put into the block they're working on as long as no other transaction has spent that output either previously in the blockchain, or in the same block.  The in-memory transaction pool can technically be managed in whatever way the miner is willing to implement.</ref>
# For each input, look in the main branch and the transaction pool to find the referenced output transaction. If the output transaction is missing for any input, this will be an orphan transaction. Add to the orphan transactions, if a matching transaction is not in there already.
# For each input, look in the main branch and the transaction pool to find the referenced output transaction. If the output transaction is missing for any input, this will be an orphan transaction. Add to the orphan transactions, if a matching transaction is not in there already.
# For each input, if we are using the ''n''th output of the earlier transaction, but it has fewer than n+1 outputs, reject this transaction
# For each input, if the referenced output transaction is coinbase (i.e. only 1 input, with hash=0, n=-1), it must have at least COINBASE_MATURITY (100) confirmations; else reject this transaction
# For each input, if the referenced output transaction is coinbase (i.e. only 1 input, with hash=0, n=-1), it must have at least COINBASE_MATURITY confirmations; else reject this transaction
# For each input, if the referenced output does not exist (e.g. never existed or has already been spent), reject this transaction<ref>This is the protection against double-spending</ref>
# Verify crypto signatures for each input; reject if any are bad
# For each input, if the referenced output has already been spent by a transaction in the main branch or the transaction pool, reject this transaction
# Using the referenced output transactions to get input values, check that each input value, as well as the sum, are in legal money range
# Using the referenced output transactions to get input values, check that each input value, as well as the sum, are in legal money range
# Reject if the sum of input values < sum of output values
# Reject if the sum of input values < sum of output values
# Reject if transaction fee (defined as sum of input values minus sum of output values) would be too low to get into an empty block
# Reject if transaction fee (defined as sum of input values minus sum of output values) would be too low to get into an empty block
# Add to transaction pool
# Verify the [[Script|scriptPubKey]] accepts for each input; reject if any are bad
# Add to wallet if mine
# Add to transaction pool<ref>Note that when the transaction is accepted into the memory pool, an additional check is made to ensure that the coinbase value does not exceed the transaction fees plus the expected BTC value (25BTC as of this writing).</ref>
# "Add to wallet if mine"
# Relay transaction to peers
# Relay transaction to peers
# For each orphan transaction that uses this one as one of its inputs, run all these steps (including this one) recursively on that orphan
# For each orphan transaction that uses this one as one of its inputs, run all these steps (including this one) recursively on that orphan


===Explanation of Some Rules===
Most rules are self-explanatory. This section explains why some of the less obvious rules are in place.


== "block" messages ==
== [[Protocol_specification#block|"block"]] messages ==


These messages hold a single block.
These messages hold a single [[block]].


# Check syntactic correctness
# Check syntactic correctness
Line 78: Line 105:
# For each transaction, apply "tx" checks 2-4
# For each transaction, apply "tx" checks 2-4
# For the coinbase (first) transaction, scriptSig length must be 2-100
# For the coinbase (first) transaction, scriptSig length must be 2-100
# For the other transactions, reject if any input has hash=0, n=-1
# Reject if sum of transaction sig opcounts > MAX_BLOCK_SIGOPS
# Reject if sum of transaction sig opcounts > MAX_BLOCK_SIGOPS
# Verify Merkle hash
# Verify Merkle hash
# Check if prev block (matching ''prev'' hash) is in main branch or side branches. If not, add this to orphan blocks, then query peer we got this from for 1st missing orphan block in ''prev'' chain; done with block
# Check if prev block (matching ''prev'' hash) is in main branch or side branches. If not, add this to orphan blocks, then query peer we got this from for 1st missing orphan block in ''prev'' chain; done with block
# Check that ''nBits'' value matches the difficulty rules
# Check that ''nBits'' value matches the difficulty rules
# Reject if timestamp is before prev block (for a complicated definition of "before")
# Reject if timestamp is the median time of the last 11 blocks or before
# For certain blocks, on initial block download, check that hash is what it should be
# For certain old blocks (i.e. on initial block download) check that hash matches known values
# Add block into the tree. There are three cases: 1. block further extends the main branch; 2. block extends a side branch but does not add enough difficulty to make it become the new main branch; 3. block extends a side branch and makes it the new main branch.
# For case 1, adding to main branch:
## For all but the coinbase transaction, apply the following:
### For each input, look in the main branch to find the referenced output transaction. Reject if the output transaction is missing for any input.
### For each input, if we are using the ''n''th output of the earlier transaction, but it has fewer than n+1 outputs, reject.
### For each input, if the referenced output transaction is coinbase (i.e. only 1 input, with hash=0, n=-1), it must have at least COINBASE_MATURITY (100) confirmations; else reject.
### Verify crypto signatures for each input; reject if any are bad
### For each input, if the referenced output has already been spent by a transaction in the main branch, reject
### Using the referenced output transactions to get input values, check that each input value, as well as the sum, are in legal money range
### Reject if the sum of input values < sum of output values
## Reject if coinbase value > sum of block creation fee and transaction fees
## (If we have not rejected):
## For each transaction, "Add to wallet if mine"
## For each transaction in the block, delete any matching transaction from the transaction pool
## Relay block to our peers
## If we rejected, the block is not counted as part of the main branch
# For case 2, adding to a side branch, we don't do anything.
# For case 3, a side branch becoming the main branch:
## Find the ''fork'' block on the main branch which this side branch forks off of
## Redefine the main branch to only go up to this ''fork'' block
## For each block on the side branch, from the child of the ''fork'' block to the leaf, add to the main branch:
### Do "branch" checks 3-11
### For all but the coinbase transaction, apply the following:
#### For each input, look in the main branch to find the referenced output transaction. Reject if the output transaction is missing for any input.
#### For each input, if we are using the ''n''th output of the earlier transaction, but it has fewer than n+1 outputs, reject.
#### For each input, if the referenced output transaction is coinbase (i.e. only 1 input, with hash=0, n=-1), it must have at least COINBASE_MATURITY (100) confirmations; else reject.
#### Verify crypto signatures for each input; reject if any are bad
#### For each input, if the referenced output has already been spent by a transaction in the main branch, reject
#### Using the referenced output transactions to get input values, check that each input value, as well as the sum, are in legal money range
#### Reject if the sum of input values < sum of output values
### Reject if coinbase value > sum of block creation fee and transaction fees
### (If we have not rejected):
### For each transaction, "Add to wallet if mine"
## If we reject at any point, leave the main branch as what it was originally, done with block
## For each block in the old main branch, from the leaf down to the child of the ''fork'' block:
### For each non-coinbase transaction in the block:
#### Apply "tx" checks 2-9, except in step 8, only look in the transaction pool for duplicates, not the main branch
#### Add to transaction pool if accepted, else go on to next transaction
## For each block in the new main branch, from the child of the ''fork'' node to the leaf:
### For each transaction in the block, delete any matching transaction from the transaction pool
## Relay block to our peers
# For each orphan block for which this block is its ''prev'', run all these steps (including this one) recursively on that orphan
 
== See Also ==
 
* [[Protocol specification]]
* [[Bitcoin Improvement Proposals]]
* [[Hardfork Wishlist]]
 
==References==
<references />


[[Category:Technical]][[Category:Developer]]
[[Category:Technical]][[Category:Developer]]

Latest revision as of 10:29, 23 June 2020

Rules for clients.

The wiki substantially documents the Bitcoin protocol, but equally important are the rules used by the client to process messages. It's crucial that clients follow certain rules in order to maintain consistency across the network, and to protect the Bitcoin security guarantees.

Here, the focus is on handling tx and block messages, because that is the tricky logic. This will skip over the method of requesting and forwarding these messages for now, and describe what to do when they are received. Also, this will describe the minimal data structures in rather abstract terms, ignoring the client's various indexes, maps and hash tables used for efficiency. This will be a conceptual description. This is all based on a fairly literal reading of the source code.

Mining (block generation) rules are not yet presented.

Data structures

The main data structures are transactions and blocks. Blocks are composed of the block header followed by transactions in the block. Transactions are identified by their hash; blocks by the hash of their header. Blocks have prev pointers that link them into a graph.

Conceptually, the client has the following data structures:

Transactions

There are two collections of transactions:

transaction pool
an unordered collection of transactions that are not in blocks in the main chain, but for which we have input transactions
orphan transactions
transactions that can't go into the pool due to one or more missing input transactions

Blocks

There are 3 categories of blocks:

blocks in the main branch
the transactions in these blocks are considered at least tentatively confirmed
blocks on side branches off the main branch
these blocks have at least tentatively lost the race to be in the main branch
orphan blocks
these are blocks which don't link into the main branch, normally because of a missing predecessor or nth-level predecessor

Blocks in the first two categories form a tree rooted at the genesis block, linked by the prev pointer, which points toward the root. (It is a very linear tree with few and short branches off the main branch.) The main branch is defined as the branch with highest total difficulty, summing the difficulties for each block in the branch.

See also Block Status.

Difficulty change

The difficulty changes every 2016 blocks. This choice is designed to occur approximately every two weeks.

2 weeks / 10 minutes = 14 * 24 * 60 / 10 = 2016

Once 2016 blocks has been reached we loop back until we hit the 2016th block before the current one. We find the difference in time between the current block and that one. This difference (called the actual timespan) is limited in bounds between [2 weeks/4, 2 weeks*4].

Then we get the last target for this old 2 week window and multiply it by the ratio of the actual timespan / the target timespan (2 weeks in secs).

new target = old target * time for 2016 blocks / 2 weeks.

If the old set of blocks completed too fast then the target is lowered (difficulty goes up) ensuring it takes longer to solve these new blocks... and vice versa. This way the difficulty oscillates around the ideal of 2 weeks (and 10 mins per block).

Block creation fee

The block creation fee changes at every 210000 blocks. The block creation fee is a function of block height on the chain (genesis=0), and is calculated using 64 bit integer operations (in satoshis) as:

(50 * 100000000) >> (height / 210000)

The block creation fee started with 50 BTC, has fallen to 25 BTC at block 210000, fell to 12.5 BTC at block 420000, to 6.25 at block 630000, will halve every 210000 blocks (roughly 4 years) and finally go down to 0 satoshi with block 6930000 around the year 2140. The block creation fee of all coinbase transactions will sum up to 2099999997690000 satoshis, practically 21million BTC.

"tx" messages

These messages hold a single transaction.

  1. Check syntactic correctness
  2. Make sure neither in or out lists are empty
  3. Size in bytes <= MAX_BLOCK_SIZE
  4. Each output value, as well as the total, must be in legal money range
  5. Make sure none of the inputs have hash=0, n=-1 (coinbase transactions)
  6. Check that nLockTime <= INT_MAX[1], size in bytes >= 100[2], and sig opcount <= 2[3]
  7. Reject "nonstandard" transactions: scriptSig doing anything other than pushing numbers on the stack, or scriptPubkey not matching the two usual forms[4]
  8. Reject if we already have matching tx in the pool, or in a block in the main branch
  9. For each input, if the referenced output exists in any other tx in the pool, reject this transaction.[5]
  10. For each input, look in the main branch and the transaction pool to find the referenced output transaction. If the output transaction is missing for any input, this will be an orphan transaction. Add to the orphan transactions, if a matching transaction is not in there already.
  11. For each input, if the referenced output transaction is coinbase (i.e. only 1 input, with hash=0, n=-1), it must have at least COINBASE_MATURITY (100) confirmations; else reject this transaction
  12. For each input, if the referenced output does not exist (e.g. never existed or has already been spent), reject this transaction[6]
  13. Using the referenced output transactions to get input values, check that each input value, as well as the sum, are in legal money range
  14. Reject if the sum of input values < sum of output values
  15. Reject if transaction fee (defined as sum of input values minus sum of output values) would be too low to get into an empty block
  16. Verify the scriptPubKey accepts for each input; reject if any are bad
  17. Add to transaction pool[7]
  18. "Add to wallet if mine"
  19. Relay transaction to peers
  20. For each orphan transaction that uses this one as one of its inputs, run all these steps (including this one) recursively on that orphan

Explanation of Some Rules

Most rules are self-explanatory. This section explains why some of the less obvious rules are in place.

"block" messages

These messages hold a single block.

  1. Check syntactic correctness
  2. Reject if duplicate of block we have in any of the three categories
  3. Transaction list must be non-empty
  4. Block hash must satisfy claimed nBits proof of work
  5. Block timestamp must not be more than two hours in the future
  6. First transaction must be coinbase (i.e. only 1 input, with hash=0, n=-1), the rest must not be
  7. For each transaction, apply "tx" checks 2-4
  8. For the coinbase (first) transaction, scriptSig length must be 2-100
  9. Reject if sum of transaction sig opcounts > MAX_BLOCK_SIGOPS
  10. Verify Merkle hash
  11. Check if prev block (matching prev hash) is in main branch or side branches. If not, add this to orphan blocks, then query peer we got this from for 1st missing orphan block in prev chain; done with block
  12. Check that nBits value matches the difficulty rules
  13. Reject if timestamp is the median time of the last 11 blocks or before
  14. For certain old blocks (i.e. on initial block download) check that hash matches known values
  15. Add block into the tree. There are three cases: 1. block further extends the main branch; 2. block extends a side branch but does not add enough difficulty to make it become the new main branch; 3. block extends a side branch and makes it the new main branch.
  16. For case 1, adding to main branch:
    1. For all but the coinbase transaction, apply the following:
      1. For each input, look in the main branch to find the referenced output transaction. Reject if the output transaction is missing for any input.
      2. For each input, if we are using the nth output of the earlier transaction, but it has fewer than n+1 outputs, reject.
      3. For each input, if the referenced output transaction is coinbase (i.e. only 1 input, with hash=0, n=-1), it must have at least COINBASE_MATURITY (100) confirmations; else reject.
      4. Verify crypto signatures for each input; reject if any are bad
      5. For each input, if the referenced output has already been spent by a transaction in the main branch, reject
      6. Using the referenced output transactions to get input values, check that each input value, as well as the sum, are in legal money range
      7. Reject if the sum of input values < sum of output values
    2. Reject if coinbase value > sum of block creation fee and transaction fees
    3. (If we have not rejected):
    4. For each transaction, "Add to wallet if mine"
    5. For each transaction in the block, delete any matching transaction from the transaction pool
    6. Relay block to our peers
    7. If we rejected, the block is not counted as part of the main branch
  17. For case 2, adding to a side branch, we don't do anything.
  18. For case 3, a side branch becoming the main branch:
    1. Find the fork block on the main branch which this side branch forks off of
    2. Redefine the main branch to only go up to this fork block
    3. For each block on the side branch, from the child of the fork block to the leaf, add to the main branch:
      1. Do "branch" checks 3-11
      2. For all but the coinbase transaction, apply the following:
        1. For each input, look in the main branch to find the referenced output transaction. Reject if the output transaction is missing for any input.
        2. For each input, if we are using the nth output of the earlier transaction, but it has fewer than n+1 outputs, reject.
        3. For each input, if the referenced output transaction is coinbase (i.e. only 1 input, with hash=0, n=-1), it must have at least COINBASE_MATURITY (100) confirmations; else reject.
        4. Verify crypto signatures for each input; reject if any are bad
        5. For each input, if the referenced output has already been spent by a transaction in the main branch, reject
        6. Using the referenced output transactions to get input values, check that each input value, as well as the sum, are in legal money range
        7. Reject if the sum of input values < sum of output values
      3. Reject if coinbase value > sum of block creation fee and transaction fees
      4. (If we have not rejected):
      5. For each transaction, "Add to wallet if mine"
    4. If we reject at any point, leave the main branch as what it was originally, done with block
    5. For each block in the old main branch, from the leaf down to the child of the fork block:
      1. For each non-coinbase transaction in the block:
        1. Apply "tx" checks 2-9, except in step 8, only look in the transaction pool for duplicates, not the main branch
        2. Add to transaction pool if accepted, else go on to next transaction
    6. For each block in the new main branch, from the child of the fork node to the leaf:
      1. For each transaction in the block, delete any matching transaction from the transaction pool
    7. Relay block to our peers
  19. For each orphan block for which this block is its prev, run all these steps (including this one) recursively on that orphan

See Also

References

  1. nLockTime must not exceed 31 bits, as some clients will interpret it incorrectly
  2. A valid transaction requires at least 100 bytes. If it's any less, the transaction is not valid
  3. The number of signature operands in the signature (no, that is not redundant) for standard transactions will never exceed two
  4. Note that this is not a hard requirement on clients.
  5. Note that this is not a hard requirement on clients. The network-enforced rule is that only one transaction spending a particular output can be in the blockchain, thus preventing double-spending. Technically miners can choose which one they want to put into the block they're working on as long as no other transaction has spent that output either previously in the blockchain, or in the same block. The in-memory transaction pool can technically be managed in whatever way the miner is willing to implement.
  6. This is the protection against double-spending
  7. Note that when the transaction is accepted into the memory pool, an additional check is made to ensure that the coinbase value does not exceed the transaction fees plus the expected BTC value (25BTC as of this writing).