BIP 0022

This page describes a BIP (Bitcoin Improvement Proposal). Please see BIP 2 for more information about BIPs and creating them. Please do not just create a wiki page.

  BIP: 22
  Title: getmemorypool
  Author: Luke Dashjr <luke+bip22@dashjr.org>
  Status: Draft
  Type: Standards Track
  Created: 28-02-2012

Abstract

This BIP describes a new JSON-RPC method for "smart" Bitcoin miners and proxies. Instead of sending a simple block header for hashing, the entire block structure is sent, and left to the miner to (optionally) customize and assemble.

Specification

JSON-RPC Method: getmemorypool

A new JSON-RPC method is defined, called "getmemorypool". It accepts two arguments, both optional. If the first argument is provided, and a String, it is interpreted as block data submission or proposal; otherwise, this call is interpreted as a request for a new block template from the server. The second argument is an Object of request parameters.

Block Template Request

If called with a single Object argument, getmemorypool interprets that Object as a set of parameters to build a block template from:

Key	Type	Description
longpollid	String	"longpollid" of job to monitor for expiration; required and valid only for long poll requests
nonces	Number	size of nonce range the miner needs; if not provided, the server SHOULD assume the client requires 232
sigoplimit	Number	number of sigops to include in template, at most; if negative, offset from the block maximum
sizelimit	Number	number of bytes to use for the entire block, at most; if negative, offset from the block maximum
tx	String	format of response "transactions" key elements ("hex" per default)

getmemorypool MUST return a JSON Object containing the following keys:

Key	Required	Type	Description
bits	Yes	String	the compressed difficulty in hexadecimal
curtime	Yes	Number	the current time as seen by the server (recommended for block time)
height	Should	Number	the height of the block we are looking for
previousblockhash	Yes	String	the hash of the previous block, in big-endian hexadecimal
sigoplimit	Should	Number	number of sigops allowed in blocks
sizelimit	Should	Number	number of bytes allowed in blocks
transactions ("hash")	Should	Array of Strings	hashes/ids of Bitcoin transactions (excluding coinbase), in little-endian hexadecimal
transactions ("hex")	Should	Array of Strings	data for Bitcoin transactions (excluding coinbase) encoded in hexadecimal (byte-for-byte)
transactions ("obj")	Should	Array of Objects	Objects containing information for Bitcoin transactions (excluding coinbase)
version	Yes	Number	always 1 (at least for bitcoin)
coinbasetxn	or ↓	See "transactions"	coinbase transaction, in the same format as "transactions" key
coinbasevalue	or ↑	Number	total funds available for the coinbase (in Satoshis)
coinbaseaux	No	Object	data that SHOULD or MUST (depending on mutable flags) be included in the coinbase's scriptSig content. Only the values (hexadecimal byte-for-byte) in this Object should be included, not the keys.
expires	No	Number	how many seconds (beginning from when the server sent the response) this work is valid for, at most
fulltarget	No	String	the number which full results should be less than, in big-endian hexadecimal (see "share/*" mutations)
longpoll	No	String	URI for long polling
longpollid	if ↑	String	unique identifier for long poll request
maxtime	No	Number	the maximum time allowed
maxtimeoff	No	Number	the maximum time allowed (as a moving offset from "curtime" - every second, the actual maxtime is incremented by 1; for example, "maxtimeoff":0 means "time" may be incremented by 1 every second)
mintime	No	Number	the minimum time allowed
mintimeoff	No	Number	the minimum time allowed (as a moving offset from "curtime")
mutable	No	Array of Strings	different manipulations that the server explicitly allows to be made (see Mutations)
noncerange	No	String	two 32-bit integers, concatenated in big-endian hexadecimal, which represent the valid ranges of nonces the miner may scan
submitmodes	No	Array of Strings	MAY contain "proposal" to indicate support for block proposal
submitold	No	Boolean	only relevant for long poll responses; indicates if work received prior to this response remains potentially valid (default) and should have its shares submitted; if false, the miner may wish to discard its share queue
target	No	String	the number which valid results must be less than, in big-endian hexadecimal
txrequired	No	Number	this many of the first transactions provided must be present in the final block, even if the "transactions/remove" mutation is allowed
workid	No	String or Number	if provided, this value must be returned with results (see "submitblock")

Transactions Object Format

When the "tx" option is provided as "obj", the Objects listed in the response's "transactions" key may contain these keys:

Key	Required	Type	Description
data	Yes	String	transaction data encoded in hexadecimal (byte-for-byte)
depends	No	Array of Numbers	other transactions (by index in "transactions" list) that must be present in the final block if this one is
fee	No	Number	difference in value between transaction inputs and outputs (in Satoshis); for coinbase transactions, this is a negative Number of the total collected block fees (ie, not including the block subsidy)
hash	No	String	hash/id encoded in little-endian hexadecimal
required	No	Boolean	if provided and true, this transaction must be in the final block, even if the "transactions/remove" mutation is allowed
sigops	No	Number	total number of SigOps, as counted for purposes of block limits
sigopsreal	No	Number	total number of SigOps, as counted by current standard rules

Mutations

Any of these may be listed in the "mutable" key to signify modifications the miner is allowed to make:

Value	Significance
coinbase/append	append the provided coinbase scriptSig
coinbase	provide their own coinbase; if one is provided, it may be replaced or modified (implied if "coinbasetxn" omitted)
generation	add or remove outputs from the coinbase/generation transaction (implied if "coinbasetxn" omitted)
share/coinbase	if the block hash is less than "target", but not less than "fulltarget", only return the block header and coinbase transaction, but only if the other transactions are unmodified from those proposed in the getmemorypool job
share/merkle	if the block hash is less than "target", but not less than "fulltarget", only return the block header, coinbase transaction, and merkle tree connecting that transaction to the root (in the form of repeated right-side SHA256 hashes) in "submitblock"
share/truncate	if the block hash is less than "target", but not less than "fulltarget", only return the block header in "submitblock"
time/increment	change the time header to a value after "time" (implied if "maxtime" or "maxtimeoff" are provided)
time/decrement	change the time header to a value before "time" (implied if "mintime" is provided)
time	modify the time header of the block
transactions/add	add other valid transactions to the block
transactions/remove	remove transactions provided by the server
transactions	add or remove transactions (both of the above; implied if "transactions" omitted from result)
prevblock	use the work with other previous-blocks; this implicitly allows removing transactions that are no longer valid, unless they are part of the "txrequired" count; it also implies adjusting "height" as necessary

Note that miners are NOT required to implement any of these mutations.

Long Polling

If the server supports long polling, it SHOULD include the "longpoll" key with a relative or absolute URI. The absolute URI MAY specify a different port than the original connection. If "longpoll" is provided, a "longpollid" key MUST also be, with a unique identifier used to determine expiration of the current job. Miners MAY start a request to this long polling URI with a standard JSON-RPC request (in the case of HTTP transport, POST with data) and same authorization. The miner MUST include the "longpollid" key provided in the parameters to this getmemorypool call. This request SHOULD NOT be processed nor answered by the server until it wishes to replace the current block data as identified by the "longpollid". Clients SHOULD make this request with a very long request timeout and MUST accept the server sending response headers with "chunked" Transfer-Encoding, only delaying the completion of the final JSON response.

Upon receiving a completed response:

• Only if "submitold" is provided and false, the client MAY discard the results of past operations and MUST begin working on the new work immediately.
• The client SHOULD begin working on the new work received as soon as possible, if not immediately.
• The client SHOULD make a new request to the same long polling URI.

If a client receives an incomplete or invalid response, it SHOULD retry the request with an exponential backoff. Clients MAY implement this backoff with limitations (such as maximum backoff time) or any algorithm as deemed suitable. It is however forbidden to simply retry immediately with no delay after more than one failure. In the case of a "Forbidden" response (for example, HTTP 403), a client SHOULD NOT attempt to retry without user intervention.

Block Proposal

At any point before a job expires, a miner MAY propose a block to the server for general validation. This is accomplished by calling getmemorypool with the block data as the first argument (as in a submission), and an Object as the second argument with the "mode" key set to "proposal".

The block data MUST be validated and checked against the server's usual acceptance rules (excluding the check for a valid proof-of-work). If it is found to be in violation of any of these rules, the server MUST return a String giving the reason for the rejection (see Block Submission for example reasons). If it is acceptable as-is, the server MAY return a true Boolean. Otherwise, the server may return a new block template (with or without changes); in this case, any missing keys are assumed to default to those in the proposed block or, if not applicable, the original block template it was based on.

It is RECOMMENDED that servers which merely need to track the proposed block for later share/* submissions, return a simple Object of the form:

{"workid":"new workid"}

Clients SHOULD assume their proposed block will remain valid if the only changes they make are to the portion of the coinbase scriptSig they themselves provided (if any) and the time header. Servers SHOULD NOT break this assumption without good cause.

Block Submission

When getmemorypool is called with a String as its first argument, it is interpreted as data for a potential block; this may be truncated or merkle-ified depending on the "share/truncate" or "share/merkle" mutations, respectively. The second argument, if provided, is an Object of parameters for the submission. If this includes the "mode" key set to "proposal", the block proposal process should be followed instead. The only other defined key for this Object is the "workid" provided by the server: if a "workid" was specified in the template, it must be submitted with the share/block.

If the parameters argument is specified, this method MUST return either null (when a share is accepted), a String describing briefly the reason the share was rejected, or an Object of these with a key for each merged-mining chain the share was submitted to.

If the parameters argument is omitted, this method SHOULD return either true (if the share is accepted) or false (if rejected). Clients MUST NOT depend on this boolean response, and servers MAY choose not to implement this paragraph.

Possible reasons a share may be rejected include, but are not limited to:

Reason	Description
bad-cb-flag	the server detected a feature-signifying flag that it does not allow
bad-cb-length	the coinbase was too long (bitcoin limit is 100 bytes)
bad-cb-prefix	the server only allows appending to the coinbase, but it was modified beyond that
bad-diffbits	"bits" were changed
bad-prevblk	the previous-block is not the one the server intends to build on
bad-txnmrklroot	the block header's merkle root did not match the transaction merkle tree
bad-txns	the server didn't like something about the transactions in the block
bad-version	the version was wrong
duplicate	the server already processed this block data
high-hash	the block header did not hash to a value lower than the specified target
rejected	a generic rejection without details
stale-prevblk	the previous-block is no longer the one the server intends to build on
stale-work	the work this block was based on is no longer accepted
time-invalid	the time was not acceptable
time-too-new	the time was too far in the future
time-too-old	the time was too far in the past
unknown-user	the user submitting the block was not recognized
unknown-work	the template or workid could not be identified

Known Bugs

This BIP should not be promoted from Draft status until these are addressed:

• Longpolling currently assumes URI-based requests, which is not implied by JSON-RPC.
• (feel free to add issues here)

Motivation

There is reasonable concerns about mining currently being too centralized on pools, and the amount of control these pools hold. By exposing the details of the block proposals to the miners, they are enabled to audit and possibly modify the block before hashing it.

There is also a very minor load reduction on the pool server, since it does not need to calculate SHA256 for the block's merkle tree.

Reference Implementation

• Eloipool
• bitcoind