Difference between revisions of "Talk:Protocol documentation"

From Bitcoin Wiki
Jump to: navigation, search
(Error?: new section)
m (Gmaxwell moved page Talk:Protocol specification to Talk:Protocol documentation: Specification creates confusion that the document is proscriptive rather than descriptive)
 
(17 intermediate revisions by 6 users not shown)
Line 1: Line 1:
= Hexdumps =
 
I'm adding some hexdumps of messages and data structures and descriptions of how these are interpreted in order to help others understand these protocol and data structures.  It's a bit redundant, so any ideas as to how to remove some of the redundancy while keeping the knowledge accessible are welcome (feel free to make the changes yourself!). -- [[User:X6763|X6763]]
 
 
= Checksum wrong? =
 
Is the checksum in the addr sample correct?  Here's what I get using OpenSSL (sorry for long lines):
 
SHA256("\x01\xe2\x15\x10\x4d\x01\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\xff\xff\x0a\x00\x00\x01\x20\x8d") =
 
d6 88 65 c8 20 61 d5 e2 54 52 b5 5b 52 17 98 b1 11 50 85 96 2e 49 e8 fd da b7 f4 fb a3 9c d8 2c
 
and SHA256 of that is
 
ed 52 39 9b 56 8e d8 d5 9a 83 72 9c 11 6f 87 d0 be f2 84 e9 98 f3 47 7c 98 61 16 9a b1 2e ed 5c
 
It could easily be I'm using OpenSSL incorrectly, so wanted to get confirmation -- [[User:AndyParkins|AndyParkins]]
 
- The checksum for "addr" was not working for me neither, and I had the same results "ED 52 39 9B", so I fixed it on the page. --[[User:Robert|Robert]]
 
 
 
= What is "77x"? Header size wrong? =
 
= What is "77x"? Header size wrong? =
 
The description for the headers command says "77x?" as the size for the block_headers[] returned.  However, the description of the block_header structure is 81 bytes (4+32+32+4+4+4+1).  What exactly is returned by the headers command? -- [[User:AndyParkins|AndyParkins]]
 
The description for the headers command says "77x?" as the size for the block_headers[] returned.  However, the description of the block_header structure is 81 bytes (4+32+32+4+4+4+1).  What exactly is returned by the headers command? -- [[User:AndyParkins|AndyParkins]]
 
= Endianess on magic numbers =
 
The given magic numbers are the wrong way around.  The magic numbers are little endian 32 bit numbers on the network, so Testnet.Magic = 0xdab5bffa and Prodnet.Magic = 0xd9b4bef9.  The two examples are the order they come in from the wire, so at the very least should be shown with spaces between the bytes.
 
[[User:AndyParkins|AndyParkins]]
 
 
:I stumbled upon this as well and corrected the page. --[[User:Theo|Theo]] 10:13, 7 June 2011 (GMT)
 
 
= Version not advertised? =
 
Doing some experiments the version seems to be sent by the client, not the server as I think the page currently says.
 
"When a node receives an incoming connection, it will immediately advertise its version."
 
That is not true. --[[User:Bluecmd|Bluecmd]] 10:02, 5 June 2011 (GMT)
 
 
:You're right, net.h says in CNode constructor: --[[User:Theo|Theo]] 10:09, 7 June 2011 (GMT)
 
        // Be shy and don't send version until we hear
 
        if (!fInbound)
 
            PushVersion();
 
  
 
= Version in getblocks? =
 
= Version in getblocks? =
Line 36: Line 8:
  
 
Does anyone else think it may be a good idea to put the hexdumps on a separate page?  It'd be nice to have this page just describe the protocol, but at the same time, be able to have examples of each of the different messages and objects.  On that node, would it also be helpful if I were to do a hexdump example of all of the different commands/structures? --[[User:Andrew12|Andrew12]] 23:26, 9 June 2011 (GMT)
 
Does anyone else think it may be a good idea to put the hexdumps on a separate page?  It'd be nice to have this page just describe the protocol, but at the same time, be able to have examples of each of the different messages and objects.  On that node, would it also be helpful if I were to do a hexdump example of all of the different commands/structures? --[[User:Andrew12|Andrew12]] 23:26, 9 June 2011 (GMT)
 +
 +
:I think that a page devoted solely to hexdumps wouldn't be a bad idea, especially if there would be multiple variants per each example and some of them would be explained step-by-step. This page needs cleanup.
 +
--[[User:ThePiachu|ThePiachu]] 14:06, 2 October 2011 (GMT)
  
 
= Proposing additional protocol messages =
 
= Proposing additional protocol messages =
Line 54: Line 29:
 
[[User:Prof7bit|Prof7bit]] 21:03, 24 June 2011 (GMT)
 
[[User:Prof7bit|Prof7bit]] 21:03, 24 June 2011 (GMT)
  
== Error? ==
+
== Errors? ==
  
 
https://en.bitcoin.it/wiki/Protocol_specification#version
 
https://en.bitcoin.it/wiki/Protocol_specification#version
  
Under version message there is the field version which has the type of int32_t. Shouldn't it be uint32_t?
+
Under version message there is the field version which has the type of int32_t. Shouldn't it be uint32_t? Same with the start_height.
 +
--[[User:ThePiachu|ThePiachu]] 18:04, 30 September 2011 (GMT)
 +
 
 +
The modern example (protocol 60002) of the version message appears to have an incorrect checksum. When running the same data through a checksum I get 3B 64 8D 5A. Here's the output of the whole message in the same blocks as in the page:
 +
<pre>
 +
F9 BE B4 D9
 +
76 65 72 73 69 6F 6E 00 00 00 00 00
 +
64 00 00 00
 +
3B 64 8D 5A
 +
62 EA 00 00
 +
01 00 00 00 00 00 00 00
 +
11 B2 D0 50 00 00 00 00
 +
01 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 FF FF 00 00 00 00 00 00
 +
01 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 FF FF 00 00 00 00 00 00
 +
3B 2E B3 5D 8C E6 17 65
 +
0F 2F 53 61 74 6F 73 68 69 3A 30 2E 37 2E 32 2F
 +
C0 3E 03 00
 +
</pre>
 +
--[[User:Papercrane|papercrane]] 12:29, 7 February 2013 (PST)
 +
 
 +
https://en.bitcoin.it/wiki/Protocol_specification#Network_address
 +
 
 +
The hexdump of the network address doesn't seem to have the time field.
 +
--[[User:ThePiachu|ThePiachu]] 18:04, 30 September 2011 (GMT)
 +
:Are you looking at the network address in a version packet? Are you using an up-to-date Bitcoin client? --[[User:Atheros|Atheros]] 22:33, 2 October 2011 (GMT)
 +
 
 +
https://en.bitcoin.it/wiki/Protocol_specification#getblocks
 +
 
 +
Shouldn't the field size of block locator hashes be 32x? ?
 +
--[[User:ThePiachu|ThePiachu]] 21:35, 1 October 2011 (GMT)
 +
 
 +
https://en.bitcoin.it/wiki/Protocol_specification#getheaders
 +
 
 +
Shouldn't the field size of hash_start be 32 ?
 +
--[[User:ThePiachu|ThePiachu]] 21:35, 1 October 2011 (GMT)
 +
 
 +
https://en.bitcoin.it/wiki/Protocol_specification#headers
 +
 
 +
I think the Field Size of Block Headers is wrong (77x? rather than 81x?).
 +
--[[User:ThePiachu|ThePiachu]] 18:45, 7 October 2011 (GMT)
 +
 
 +
== Missing? ==
 +
 
 +
The nonce in the version message is created when the local node is created, not when connection is established. I still have to confirm this in a real example before correcting it in the wiki.
 +
[[User:Jdnavarro|Jdnavarro]] ([[User talk:Jdnavarro|talk]]) 16:15, 12 September 2012 (GMT)
 +
 
 +
It seems that the magic bytes for the testnet in version 0.7.0 is 0x0709110B. Am I missing something?
 +
[[User:Jdnavarro|Jdnavarro]] ([[User talk:Jdnavarro|talk]]) 12:28, 23 September 2012 (GMT)
 +
 
 +
When a '''getdata''' message with multiple blocks is sent (i.e. initial block download), the remote node, after sending all the requested block messages, it sends an '''inv''' message with its top block.
 +
[[User:Jdnavarro|Jdnavarro]] ([[User talk:Jdnavarro|talk]]) 16:08, 8 October 2012 (GMT)
 +
 
 +
The maximum size for a Block or a Transaction is 1MB. The overall message size is 32MB, which I think is deliberately redundant because it shouldn't be possible to generate a message of such size with the other limits. Should this be part of the protocol?
 +
[[User:Jdnavarro|Jdnavarro]] ([[User talk:Jdnavarro|talk]]) 18:20, 15 October 2012 (GMT)
 +
 
 +
== txn_count ==
 +
 
 +
In [[Header#Block_Headers]], it says txn_count is always zero. Why is this? --[[User:Rebroad|Rebroad]] ([[User talk:Rebroad|talk]]) 03:32, 10 September 2014 (UTC)
 +
 
 +
: IIRC, it's because the txn_count is the number of transactions ''actually'' being sent - which in the header request is zero because you're only getting the headers. --[[User:Luke-jr|Luke-jr]] ([[User talk:Luke-jr|talk]]) 05:23, 10 September 2014 (UTC)

Latest revision as of 19:31, 25 January 2015

What is "77x"? Header size wrong?

The description for the headers command says "77x?" as the size for the block_headers[] returned. However, the description of the block_header structure is 81 bytes (4+32+32+4+4+4+1). What exactly is returned by the headers command? -- AndyParkins

Version in getblocks?

Apparently the official client sends the protocol version in getblocks messages, (possibly even in getheaders). This seems to me to be just weird --Bluecmd 14:57, 6 June 2011 (GMT)

Separate page for hexdumps?

Does anyone else think it may be a good idea to put the hexdumps on a separate page? It'd be nice to have this page just describe the protocol, but at the same time, be able to have examples of each of the different messages and objects. On that node, would it also be helpful if I were to do a hexdump example of all of the different commands/structures? --Andrew12 23:26, 9 June 2011 (GMT)

I think that a page devoted solely to hexdumps wouldn't be a bad idea, especially if there would be multiple variants per each example and some of them would be explained step-by-step. This page needs cleanup.

--ThePiachu 14:06, 2 October 2011 (GMT)

Proposing additional protocol messages

According to the Bitcoin paper in order to do the simplified verification the client needs to store all headers of the entire chain but needs data only for those blocks that contain transactions of interest.

For this I propose the following new message types:

  • setfilter this message contains a list of bitcoin addresses. After receiving this message on a connection the client will stop broadcasting any inv messages for transactions which don't have a matching address in its inputs or outputs. inv for new blocks will not be filtered.
  • getblocksfiltered this message works exactly like getblocks but it will be answered with a filtered list of only those blocks that contain transactions with unspent outputs matching the filter list.

and the following new inventory type for getdata

  • MSG_BLOCK_PRUNED this will be answered like a MSG_BLOCK but all transactions not matching the filter list and transactions that are already spent will be stripped from the block before transmitting it to the client.

When a client requests such filtering then all new blocks will still be advertised like normal but the client will have the opportunity to request only the absolute minimum necessary data to perform the simplified verification as outlined in the paper and new transactions will only be sent to it if they match the filter list. This should save a LOT of bandwidth. Prof7bit 21:03, 24 June 2011 (GMT)

Errors?

https://en.bitcoin.it/wiki/Protocol_specification#version

Under version message there is the field version which has the type of int32_t. Shouldn't it be uint32_t? Same with the start_height. --ThePiachu 18:04, 30 September 2011 (GMT)

The modern example (protocol 60002) of the version message appears to have an incorrect checksum. When running the same data through a checksum I get 3B 64 8D 5A. Here's the output of the whole message in the same blocks as in the page:

F9 BE B4 D9
76 65 72 73 69 6F 6E 00 00 00 00 00
64 00 00 00
3B 64 8D 5A
62 EA 00 00
01 00 00 00 00 00 00 00
11 B2 D0 50 00 00 00 00
01 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 FF FF 00 00 00 00 00 00
01 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 FF FF 00 00 00 00 00 00
3B 2E B3 5D 8C E6 17 65
0F 2F 53 61 74 6F 73 68 69 3A 30 2E 37 2E 32 2F
C0 3E 03 00

--papercrane 12:29, 7 February 2013 (PST)

https://en.bitcoin.it/wiki/Protocol_specification#Network_address

The hexdump of the network address doesn't seem to have the time field. --ThePiachu 18:04, 30 September 2011 (GMT)

Are you looking at the network address in a version packet? Are you using an up-to-date Bitcoin client? --Atheros 22:33, 2 October 2011 (GMT)

https://en.bitcoin.it/wiki/Protocol_specification#getblocks

Shouldn't the field size of block locator hashes be 32x? ? --ThePiachu 21:35, 1 October 2011 (GMT)

https://en.bitcoin.it/wiki/Protocol_specification#getheaders

Shouldn't the field size of hash_start be 32 ? --ThePiachu 21:35, 1 October 2011 (GMT)

https://en.bitcoin.it/wiki/Protocol_specification#headers

I think the Field Size of Block Headers is wrong (77x? rather than 81x?). --ThePiachu 18:45, 7 October 2011 (GMT)

Missing?

The nonce in the version message is created when the local node is created, not when connection is established. I still have to confirm this in a real example before correcting it in the wiki. Jdnavarro (talk) 16:15, 12 September 2012 (GMT)

It seems that the magic bytes for the testnet in version 0.7.0 is 0x0709110B. Am I missing something? Jdnavarro (talk) 12:28, 23 September 2012 (GMT)

When a getdata message with multiple blocks is sent (i.e. initial block download), the remote node, after sending all the requested block messages, it sends an inv message with its top block. Jdnavarro (talk) 16:08, 8 October 2012 (GMT)

The maximum size for a Block or a Transaction is 1MB. The overall message size is 32MB, which I think is deliberately redundant because it shouldn't be possible to generate a message of such size with the other limits. Should this be part of the protocol? Jdnavarro (talk) 18:20, 15 October 2012 (GMT)

txn_count

In Header#Block_Headers, it says txn_count is always zero. Why is this? --Rebroad (talk) 03:32, 10 September 2014 (UTC)

IIRC, it's because the txn_count is the number of transactions actually being sent - which in the header request is zero because you're only getting the headers. --Luke-jr (talk) 05:23, 10 September 2014 (UTC)