BIP 0022
|
This page describes a BIP (Bitcoin Improvement Proposal). |
BIP: 22 Title: getmemorypool Author: Luke Dashjr <luke+bip22@dashjr.org> Status: Draft Type: Standards Track Created: 28-02-2012
Abstract
This BIP describes a new JSON-RPC method for "smart" Bitcoin miners and proxies. Instead of sending a simple block header for hashing, the entire block structure is sent, and left to the miner to (optionally) customize and assemble.
Specification
JSON-RPC Method: getmemorypool
A new JSON-RPC method is defined, called "getmemorypool". It accepts two arguments, both optional. If the first argument is provided, and a String, it is interpreted as block data for a submission or proposal call; otherwise, this call is interpreted as a request for a new block template from the server. The second argument is an Object of request parameters.
Block Template Request
getmemorypool({parameters...})
If called with a single Object argument, getmemorypool interprets that Object as a set of parameters to build a block template from:
| Key | Type | Description |
|---|---|---|
| capabilities | Array of Strings | SHOULD contain a list of the following, to indicate client-side support: "longpoll", "coinbasetxn", "coinbasevalue", "proposal", "serverlist", "workid", and any of the mutations |
| longpoll | Boolean | if provided and true, indicates this is a long poll request |
| longpollid | String | "longpollid" of job to monitor for expiration; required and valid only for long poll requests |
| nonces | Number | size of nonce range the miner needs; if not provided, the server SHOULD assume the client requires 232 |
| sigoplimit | Number | number of sigops to include in template, at most; if negative, offset from the block maximum |
| sizelimit | Number | number of bytes to use for the entire block, at most; if negative, offset from the block maximum |
| tx | String | format of response "transactions" key elements ("hex" per default) |
getmemorypool MUST return a JSON Object containing the following keys:
| Key | Required | Type | Description |
|---|---|---|---|
| bits | Yes | String | the compressed difficulty in hexadecimal |
| curtime | Yes | Number | the current time as seen by the server (recommended for block time) |
| height | Should | Number | the height of the block we are looking for |
| longpollid | Should | String | unique identifier for long poll request; MUST be omitted if the server does not support long polling |
| previousblockhash | Yes | String | the hash of the previous block, in big-endian hexadecimal |
| sigoplimit | Should | Number | number of sigops allowed in blocks |
| sizelimit | Should | Number | number of bytes allowed in blocks |
| transactions ("hash") | Should | Array of Strings | hashes/ids of Bitcoin transactions (excluding coinbase), in little-endian hexadecimal |
| transactions ("hex") | Should | Array of Strings | data for Bitcoin transactions (excluding coinbase) encoded in hexadecimal (byte-for-byte) |
| transactions ("obj") | Should | Array of Objects | Objects containing information for Bitcoin transactions (excluding coinbase) |
| version | Yes | Number | always 1 (at least for bitcoin) |
| coinbasetxn | this or ↓ | See "transactions" | coinbase transaction, in the same format as "transactions" key |
| coinbasevalue | this or ↑ | Number | total funds available for the coinbase (in Satoshis) |
| capabilities | No | Array of Strings | MAY contain "proposal" to indicate support for block proposal |
| coinbaseaux | No | Object | data that SHOULD or MUST (depending on mutable flags) be included in the coinbase's scriptSig content. Only the values (hexadecimal byte-for-byte) in this Object should be included, not the keys. |
| expires | No | Number | how many seconds (beginning from when the server sent the response) this work is valid for, at most |
| fulltarget | No | String | the number which full results should be less than, in big-endian hexadecimal (see "share/*" mutations) |
| longpolluri | No | String | if provided, an alternate URI to use for longpoll requests |
| maxtime | No | Number | the maximum time allowed |
| maxtimeoff | No | Number | the maximum time allowed (as a moving offset from "curtime" - every second, the actual maxtime is incremented by 1; for example, "maxtimeoff":0 means "time" may be incremented by 1 every second) |
| mintime | No | Number | the minimum time allowed |
| mintimeoff | No | Number | the minimum time allowed (as a moving offset from "curtime") |
| mutable | No | Array of Strings | different manipulations that the server explicitly allows to be made (see Mutations) |
| noncerange | No | String | two 32-bit integers, concatenated in big-endian hexadecimal, which represent the valid ranges of nonces the miner may scan |
| serverlist | No | Array of Objects | list of servers in this single logical service |
| submitold | No | Boolean | only relevant for long poll responses; indicates if work received prior to this response remains potentially valid (default) and should have its shares submitted; if false, the miner may wish to discard its share queue |
| target | No | String | the number which valid results must be less than, in big-endian hexadecimal |
| txrequired | No | Number | this many of the first transactions provided must be present in the final block, even if the "transactions/remove" mutation is allowed |
| workid | No | String or Number | if provided, this value must be returned with results (see "submitblock") |
Transactions Object Format
When the "tx" option is provided as "obj", the Objects listed in the response's "transactions" key may contain these keys:
| Key | Required | Type | Description |
|---|---|---|---|
| data | Yes | String | transaction data encoded in hexadecimal (byte-for-byte) |
| depends | No | Array of Numbers | other transactions (by 1-based index in "transactions" list) that must be present in the final block if this one is; if key is not present, dependencies are unknown and clients MUST NOT assume there aren't any |
| fee | No | Number | difference in value between transaction inputs and outputs (in Satoshis); for coinbase transactions, this is a negative Number of the total collected block fees (ie, not including the block subsidy); if key is not present, fee is unknown and clients MUST NOT assume there isn't one |
| hash | No | String | hash/id encoded in little-endian hexadecimal |
| required | No | Boolean | if provided and true, this transaction must be in the final block, even if the "transactions/remove" mutation is allowed |
| sigops | No | Number | total number of SigOps, as counted for purposes of block limits; if key is not present, sigop count is unknown and clients MUST NOT assume there aren't any |
| sigopsreal | No | Number | total number of SigOps, as counted by current standard rules; if key is not present, sigop count is unknown and clients MUST NOT assume there aren't any |
Mutations
Any of these may be listed in the "mutable" key to signify modifications the miner is allowed to make:
| Value | Significance |
|---|---|
| coinbase/append | append the provided coinbase scriptSig |
| coinbase | provide their own coinbase; if one is provided, it may be replaced or modified (implied if "coinbasetxn" omitted) |
| generation | add or remove outputs from the coinbase/generation transaction (implied if "coinbasetxn" omitted) |
| share/coinbase | if the block hash is less than "target", but not less than "fulltarget", only return the block header and coinbase transaction, but only if the other transactions are unmodified from those proposed in the getmemorypool job |
| share/merkle | if the block hash is less than "target", but not less than "fulltarget", only return the block header, coinbase transaction, and merkle tree connecting that transaction to the root (in the form of repeated right-side SHA256 hashes) in "submitblock" |
| share/truncate | if the block hash is less than "target", but not less than "fulltarget", only return the block header in "submitblock" |
| time/increment | change the time header to a value after "time" (implied if "maxtime" or "maxtimeoff" are provided) |
| time/decrement | change the time header to a value before "time" (implied if "mintime" is provided) |
| time | modify the time header of the block |
| transactions/add | add other valid transactions to the block |
| transactions/remove | remove transactions provided by the server |
| transactions | add or remove transactions (both of the above; implied if "transactions" omitted from result) |
| prevblock | use the work with other previous-blocks; this implicitly allows removing transactions that are no longer valid, unless they are part of the "txrequired" count; it also implies adjusting "height" as necessary |
Note that miners are NOT required to implement any of these mutations.
The format used for submitting shares with the "share/merkle" mutation shall be the 80-byte block header, followed by the coinbase transaction, and then finally the little-endian SHA256 hashes of each link in the merkle chain connecting it to the merkle root.
Long Polling
If the server supports long polling, it MUST include the "longpollid" key It MAY supply the "longpolluri" key with a relative or absolute URI, which MAY specify a completely different resource than the original connection, including port number. If "longpolluri" is provided by the server, clients MUST only attempt to use that URI for longpoll requests.
Clients MAY start a longpoll request with a standard JSON-RPC request (in the case of HTTP transport, POST with data) and same authorization, setting the "longpoll" parameter in the request to true. The client MUST also include the "longpollid" key provided by the server in the parameters.
This request SHOULD NOT be processed nor answered by the server until it wishes to replace the current block data as identified by the "longpollid". Clients SHOULD make this request with a very long request timeout and MUST accept servers sending a partial response in advance (such as HTTP headers with "chunked" Transfer-Encoding), and only delaying the completion of the final JSON response until processing.
Upon receiving a completed response:
- Only if "submitold" is provided and false, the client MAY discard the results of past operations and MUST begin working on the new work immediately.
- The client SHOULD begin working on the new work received as soon as possible, if not immediately.
- The client SHOULD make a new request to the same long polling URI.
If a client receives an incomplete or invalid response, it SHOULD retry the request with an exponential backoff. Clients MAY implement this backoff with limitations (such as maximum backoff time) or any algorithm as deemed suitable. It is, however, forbidden to simply retry immediately with no delay after more than one failure. In the case of a "Forbidden" response (for example, HTTP 403), a client SHOULD NOT attempt to retry without user intervention.
Logical Services
If the "serverlist" parameter is provided, clients MAY choose to intelligently treat the server as part of a larger single logical service.
Each host Object in the Array is comprised of the following fields:
| Key | Required | Type | Description |
|---|---|---|---|
| uri | Yes | String | URI of the individual server; if authentication information is omitted, the same authentication used for this request MUST be assumed |
| avoid | No | Number | number of seconds to avoid using this server |
| priority | No | Number | an integer priority of this host (default: 0) |
| sticky | No | Number | number of seconds to stick to this server when used |
| update | No | Boolean | whether this server may update the serverlist (default: true) |
| weight | No | Number | a relative weight for hosts with the same priority (default: 1) |
When choosing which actual server to get the next job from, URIs MUST be tried in order of their "priority" key, lowest Number first. Where the priority of URIs is the same, they should be chosen from in random order, weighed by their "weight" key.
Updates to the Logical Service server list may only be made by the original server, or servers listed with the "update" key missing or true. Clients MAY choose to advertise serverlist capability to servers with a false "update" key, but if so, MUST treat the server list provided as a subset of the current one, only considered in the context of this server. At least one server with "update" privilege MUST be attempted at least once daily.
If the "sticky" key is provided, then when that server is used, it should be used consistently for at least that many seconds, if possible.
A permanent change in server URI MAY be indicated with a simple "serverlist" parameter:
"serverlist":[{"uri": "http://newserver"}]
A temporary delegation to another server for 5 minutes MAY be indicated likewise:
"serverlist":[{"uri": "", avoid: 300}, {"uri": "http://newserver", "update": false}]
Block Proposal
getmemorypool("hex-encoded block data...", {mode: "proposal", parameters...})
At any point before a job expires, a miner MAY propose a block to the server for general validation. This is accomplished by calling getmemorypool with the hex-encoded block data as the first argument (as in a submission), and an Object as the second argument with the "mode" key set to "proposal".
The block data MUST be validated and checked against the server's usual acceptance rules (excluding the check for a valid proof-of-work). If it is found to be in violation of any of these rules, the server MUST return a String giving the reason for the rejection (see Block Submission for example reasons). If it is acceptable as-is, the server MAY return a true Boolean. Otherwise, the server may return a new block template (with or without changes); in this case, any missing keys are assumed to default to those in the proposed block or, if not applicable, the original block template it was based on.
It is RECOMMENDED that servers which merely need to track the proposed block for later share/* submissions, return a simple Object of the form:
{"workid":"new workid"}
Clients SHOULD assume their proposed block will remain valid if the only changes they make are to the portion of the coinbase scriptSig they themselves provided (if any) and the time header. Servers SHOULD NOT break this assumption without good cause.
Block Submission
getmemorypool("hex-encoded block data...", {parameters...})
When getmemorypool is called with a String as its first argument, it is interpreted as hex-encoded data for a potential block; this may be truncated or merkle-ified depending on the "share/truncate" or "share/merkle" mutations, respectively. The second argument, if provided, is an Object of parameters for the submission. If this includes the "mode" key set to "proposal", the block proposal process should be followed instead. The only other defined key for this Object is the "workid" provided by the server: if a "workid" was specified in the template, it must be submitted with the share/block.
If the parameters argument is specified, this method MUST return either null (when a share is accepted), a String describing briefly the reason the share was rejected, or an Object of these with a key for each merged-mining chain the share was submitted to.
If the parameters argument is omitted, this method SHOULD return either true (if the share is accepted) or false (if rejected). Clients MUST NOT depend on this boolean response, and servers MAY choose not to implement this paragraph.
Possible reasons a share may be rejected include, but are not limited to:
| Reason | Description |
|---|---|
| bad-cb-flag | the server detected a feature-signifying flag that it does not allow |
| bad-cb-length | the coinbase was too long (bitcoin limit is 100 bytes) |
| bad-cb-prefix | the server only allows appending to the coinbase, but it was modified beyond that |
| bad-diffbits | "bits" were changed |
| bad-prevblk | the previous-block is not the one the server intends to build on |
| bad-txnmrklroot | the block header's merkle root did not match the transaction merkle tree |
| bad-txns | the server didn't like something about the transactions in the block |
| bad-version | the version was wrong |
| duplicate | the server already processed this block data |
| high-hash | the block header did not hash to a value lower than the specified target |
| rejected | a generic rejection without details |
| stale-prevblk | the previous-block is no longer the one the server intends to build on |
| stale-work | the work this block was based on is no longer accepted |
| time-invalid | the time was not acceptable |
| time-too-new | the time was too far in the future |
| time-too-old | the time was too far in the past |
| unknown-user | the user submitting the block was not recognized |
| unknown-work | the template or workid could not be identified |
Known Bugs
This BIP should not be promoted from Draft status until these are addressed:
- (feel free to add issues here)
Motivation
There is reasonable concerns about mining currently being too centralized on pools, and the amount of control these pools hold. By exposing the details of the block proposals to the miners, they are enabled to audit and possibly modify the block before hashing it.