User:Gmaxwell/features: Difference between revisions

From Bitcoin Wiki
Jump to navigation Jump to search
Some updates, for changes which are not temporary.
→‎Proofs: formatting
 
(13 intermediate revisions by the same user not shown)
Line 2: Line 2:
This is a non-official list of features I personally would like to see in the reference Bitcoin software. Although it's just my personal list, some of these items are generally supported by other people— and I've included many things that I wouldn't use myself but think would be very good to have.
This is a non-official list of features I personally would like to see in the reference Bitcoin software. Although it's just my personal list, some of these items are generally supported by other people— and I've included many things that I wouldn't use myself but think would be very good to have.


'''Core user-facing client features'''
===Core user-facing client features===
*Type-2 deterministic wallets ([https://gist.github.com/1799467 in progress])
*Type-2 deterministic wallets ([https://gist.github.com/1799467 in progress])
*Multi-wallet (including watchmode wallets)
*Multi-wallet (including watchmode wallets)
*disk corruption robust wallet storage  
*disk corruption robust wallet storage  
*Memory/cpu error robust key generation (I have an implementation of SHA512 that is polarity reversed, all 1s are 0 and all 0s are 1 that could be used for comparison).
*Peer management RPCs (<s>peer info (last connect, version, transaction stats, blocks stats) (done)</s>, add, remove)
*Peer management RPCs (<s>peer info (last connect, version, transaction stats, blocks stats) (done)</s>, add, remove)
*<s>A pseudo-cli interface for the GUI. e.g. file->console</s> (done, 0.7.0)
*<s>A pseudo-cli interface for the GUI. e.g. file->console</s> (done, 0.7.0)
Line 17: Line 18:


*Show negative confirmations— how deep is the deepest conflicting transaction.  Allow canceling transactions which have at least -6 (say) confirmations. Easier to do safely than general cancellation.
*Show negative confirmations— how deep is the deepest conflicting transaction.  Allow canceling transactions which have at least -6 (say) confirmations. Easier to do safely than general cancellation.
**We now have negative confirmation in Bitcoin core, but it doesn't know how deep the conflict is.


*Async RPC
*Async RPC


'''Behind the scenes wallet improvements'''
===Behind the scenes wallet improvements===
*Transaction queuing (to improve coin selection and allow the use of sendmany without explicitly asking for it)
*Transaction queuing (to improve coin selection and allow the use of sendmany without explicitly asking for it)
*Smarter coin selection (minds fees!), general solver is probably too much. :) See: [[User:Gmaxwell/coin_selection]]
*Smarter coin selection (minds fees!), general solver is probably too much. :) See: [[User:Gmaxwell/coin_selection]]
Line 26: Line 28:
*Wallet sweeping (avoid jagged coins, get excess inputs destroyed when you have a chance)
*Wallet sweeping (avoid jagged coins, get excess inputs destroyed when you have a chance)


'''Features to avoid inefficient/insecure uses of the network'''
===Features to avoid inefficient/insecure uses of the network===
*Merged mining notary service, validation in client
*Merged mining notary service, validation in client
*Detached transaction signatures to prove payment/identity (avoids 'green address' privacy lost and bloat)
*Detached transaction signatures to prove payment/identity (avoids 'green address' privacy lost and bloat)
Line 32: Line 34:
*Separate network for message relay? Keep people's IM traffic out of the blockchain? (ugh)
*Separate network for message relay? Keep people's IM traffic out of the blockchain? (ugh)


'''Network layer improvements'''
===Network layer improvements===
*NTP integration (if trusted, healthy, ntp— use instead of network time)
*NTP integration (if trusted, healthy, ntp— use instead of network time)
*Better network time intelligence
*Better network time intelligence
Line 40: Line 42:
*Priority relaying (measure which nodes are most 'useful' based on blocks/txn-w-fees then relay to them before other nodes, relay to them first— move to front on good actions probably enough)
*Priority relaying (measure which nodes are most 'useful' based on blocks/txn-w-fees then relay to them before other nodes, relay to them first— move to front on good actions probably enough)
*Authenticated peers for addnode so a network attacker can't isolate you without your knowledge
*Authenticated peers for addnode so a network attacker can't isolate you without your knowledge
*Peer management RPCs (peer info (last connect, version, transaction stats, blocks stats), add, remove)
*<s>Peer management RPCs (peer info (last connect, version, transaction stats, blocks stats), add, remove)</s> (done, info in 0.7, management in 0.8)
*<s>IPv6</s> (done, 0.7.0)
*<s>IPv6</s> (done, 0.7.0)
*Onion-cat style onion peers (pull semi-ready for 0.7.0)
*<s>Onion-cat style onion peers</s> (done, 0.7.0)
*Tor integration (request its own hidden service via the control port)
*Tor integration (request its own hidden service via the control port)


*Blockchain over satellite receiver (peak average block chain datarate 14kbit/sec, sat bandwidth isn't that expensive)
*Blockchain over satellite receiver (peak average block chain datarate 14kbit/sec, sat bandwidth isn't that expensive)


'''Backend security'''
===Backend security===
*scrypt wallet encryption
*<s>scrypt wallet encryption</s>
** Removed, because I'm no longer convinced that scrypt provides a material cost increase for an attack with a amortized asic infrastructure, whos costs are power limited.
*Network sanity checks. Trusted peers down, network very slow? don't take txn out of unconfirmed.
*Network sanity checks. Trusted peers down, network very slow? don't take txn out of unconfirmed.
*timewarp fix
*timewarp fix
*More DOS/network attack hardening (time skewing, memory exhaustion, etc)
*More DOS/network attack hardening (time skewing, memory exhaustion, etc)
*Ability to mark keys as compromised and enable autosweeping of inputs that are sent to them
*Ability to mark keys as compromised and enable autosweeping of inputs that are sent to them
*Provable notice of doublespend (when a node sees a doublespend it prepares an alert indexed by the doublespent output, including at least two spends of it, and floods it, if a client receives a such a notice for a transaction sent to it, it shoots fireworks and greatly delays showing it as confirmed)
*Provable notice of doublespend (when a node sees a doublespend it prepares an alert indexed by the doublespent output, including at least two spends of it, and floods it, if a client receives a such a notice for a transaction sent to it, it shoots fireworks and greatly delays showing it as confirmed) (in progress, [
https://github.com/bitcoin/bitcoin/pull/3883])


'''Validation performance'''
===Validation performance===
*Fast chain sync (fix database stupidity) (partially completed in 0.6.0)
*<s>Fast chain sync (fix database stupidity) (partially completed in 0.6.0)</s>(0.8)
*Read only chain summaries for old history
**Compressed blockchain history, decompress to build the summary preserving zero trust
**Compressed blockchain history, decompress to build the summary preserving zero trust
*Rescan that checks all the hashes to detect and recover from on disk corruption
*<s>Rescan that checks all the hashes to detect and recover from on disk corruption</s>(0.8)
*Backwards header sync, then fill blocks, [[User:Gmaxwell/Reverse header-fetching sync|reverse header-fetching sync]]
*Backwards header sync, then fill blocks, [[User:Gmaxwell/Reverse header-fetching sync|reverse header-fetching sync]]
*Start as a SPV (lite) client, validate to become a full node in the background
*Start as a SPV (lite) client, validate to become a full node in the background


'''Mining'''
===Mining===
*P2Pool
*P2Pool
**If P2Pools have a significant amount of hashpower, and their shares carry commitments to transactions, P2Pool can create sub-minute evidence of intent to mine — coupled with double-spend-alerts can create pretty good fast confidence.
**If P2Pools have a significant amount of hashpower, and their shares carry commitments to transactions, P2Pool can create sub-minute evidence of intent to mine — coupled with double-spend-alerts can create pretty good fast confidence.
Line 76: Line 79:
*Directed graph fee/priority computation (a later txn can pay fee for parents, must avoid memory DOS)
*Directed graph fee/priority computation (a later txn can pay fee for parents, must avoid memory DOS)


'''Fee tweaks'''
===Fee tweaks===
*Consider net unspent output change (txn which decrease the unspent outputs on the network should have fewer fees)
*<s>Consider net unspent output change (txn which decrease the unspent outputs on the network should have fewer fees)</s> (Done, not quite net change as that has some weird incentives, but some of the signature size is ignored for priority calculating purposes now)
*NAK for dropped transactions so clients (sometimes) know when their txn aren't making progress
*NAK for dropped transactions so clients (sometimes) know when their txn aren't making progress
===Proofs===
If full nodes become expensive to operate in the future then they may become uncommon and this could compromise the security of Bitcoin. This risk can be reduced if it's made possible for Bitcoin nodes to check all the rules at random and transmit compact proofs of rule violations. If this is done even if there is only one honest full node in the world the system is secure so long as it can communicate to all others.
In general, in any deterministic computation process if you have simple state updates and commit to the sequence of states a compact proof of invalidity can be generated by producing a hash tree fragment to the first invalidate state transition.
Ideas in this space have been previously discussed under the banner of proof-of-treachery [https://bitcointalk.org/index.php?topic=96644.msg1064601#msg1064601].
Right now not all of the rules can be checked randomly or have compact proofs.
* SPV header checks — time, target, difficulty, already have them.  But if future SPV nodes don't really check all the headers  in the future it may be useful to arrange old header times in a merkle mountain range to proofs of sum difficulty and compact proofs of incorrect difficulty.
* Proof of invalid script
** Possible in the current system: Proof is tree fragments for the invalid txn in question as well as one invalid input (no need to include more than one).
** Could be made more efficient by including commitments to intermediate states, but with the 201 opcode limit all scripts are compact to verify in Bitcoin without doing anything fancier.
** Also proves nlocktime, etc.
** To prove output value greater than inputs all inputs must be provided in the proof.
* Proof of double spend
** Possible in the current system: Proof is tree fragments for the two transactions which spend the same input.
* Proof of false inflation
** Not possible without more data: The coinbase payment is the sum of fees in a block and the subsidy. Fees require knowing the transaction's inputs output values, to check subsidy you must not only have all the transactions but all their inputs as well.
** This is fixable. Replace the hash tree with one that H((fees_left+fees_right)||H(left)||H(right)) so that the tree commits to the fees. Nodes can randomly check this by grabbing a random txn and checking its inputs, and compactly prove violation by showing where the fees don't match their commitments. 
* Proof of block too large
** Similar to false inflation, requires all the transactions, can similarly be solved by including the sum of txn sizes in the tree.
* Proof of spending a non-existing input
** Requires additional data: proof requires scanning the entire blockchain history
** Requires a UXTO set commitment as well as a tree for outputs created and consumed within one block. Proof is a pair of tree fragments for the higher and lower records for the missing entry, and another pair for the outputs created within a block but consumed.

Latest revision as of 19:37, 8 April 2014

This is a non-official list of features I personally would like to see in the reference Bitcoin software. Although it's just my personal list, some of these items are generally supported by other people— and I've included many things that I wouldn't use myself but think would be very good to have.

Core user-facing client features

  • Type-2 deterministic wallets (in progress)
  • Multi-wallet (including watchmode wallets)
  • disk corruption robust wallet storage
  • Memory/cpu error robust key generation (I have an implementation of SHA512 that is polarity reversed, all 1s are 0 and all 0s are 1 that could be used for comparison).
  • Peer management RPCs (peer info (last connect, version, transaction stats, blocks stats) (done), add, remove)
  • A pseudo-cli interface for the GUI. e.g. file->console (done, 0.7.0)
  • Thin client mode
    • Should be able to run many bitcoin-cli bitcoin-qt instances over the network against a single trusted bitcoind network node
  • Offline wallet— BIP10ish (semidone; raw txn does this now but without good usability)
  • Ability to cancel a transaction if blocks have passed and it hasn't been mined. Canceling should put the transaction into 'hold' mode for many hours (several days?) before making its outputs available for respending, perhaps a resend-with-fee option that requires all outputs to be the same (except the change output) that can be executed faster.
  • Show negative confirmations— how deep is the deepest conflicting transaction. Allow canceling transactions which have at least -6 (say) confirmations. Easier to do safely than general cancellation.
    • We now have negative confirmation in Bitcoin core, but it doesn't know how deep the conflict is.
  • Async RPC

Behind the scenes wallet improvements

  • Transaction queuing (to improve coin selection and allow the use of sendmany without explicitly asking for it)
  • Smarter coin selection (minds fees!), general solver is probably too much. :) See: User:Gmaxwell/coin_selection
  • Better anonymity (avoid cross-linking), but multiwallet helps
  • Wallet sweeping (avoid jagged coins, get excess inputs destroyed when you have a chance)

Features to avoid inefficient/insecure uses of the network

  • Merged mining notary service, validation in client
  • Detached transaction signatures to prove payment/identity (avoids 'green address' privacy lost and bloat)
    • Payment protocol too, but it's not enough alone (in progress)
  • Separate network for message relay? Keep people's IM traffic out of the blockchain? (ugh)

Network layer improvements

  • NTP integration (if trusted, healthy, ntp— use instead of network time)
  • Better network time intelligence
  • Roaming connections (reduced attack risk, learn about peers faster)
  • Roaming inbound (when full knock off some instead of just capping)
  • Lightweight node announce to (dns)seeds?
  • Priority relaying (measure which nodes are most 'useful' based on blocks/txn-w-fees then relay to them before other nodes, relay to them first— move to front on good actions probably enough)
  • Authenticated peers for addnode so a network attacker can't isolate you without your knowledge
  • Peer management RPCs (peer info (last connect, version, transaction stats, blocks stats), add, remove) (done, info in 0.7, management in 0.8)
  • IPv6 (done, 0.7.0)
  • Onion-cat style onion peers (done, 0.7.0)
  • Tor integration (request its own hidden service via the control port)
  • Blockchain over satellite receiver (peak average block chain datarate 14kbit/sec, sat bandwidth isn't that expensive)

Backend security

  • scrypt wallet encryption
    • Removed, because I'm no longer convinced that scrypt provides a material cost increase for an attack with a amortized asic infrastructure, whos costs are power limited.
  • Network sanity checks. Trusted peers down, network very slow? don't take txn out of unconfirmed.
  • timewarp fix
  • More DOS/network attack hardening (time skewing, memory exhaustion, etc)
  • Ability to mark keys as compromised and enable autosweeping of inputs that are sent to them
  • Provable notice of doublespend (when a node sees a doublespend it prepares an alert indexed by the doublespent output, including at least two spends of it, and floods it, if a client receives a such a notice for a transaction sent to it, it shoots fireworks and greatly delays showing it as confirmed) (in progress, [

https://github.com/bitcoin/bitcoin/pull/3883])

Validation performance

  • Fast chain sync (fix database stupidity) (partially completed in 0.6.0)(0.8)
    • Compressed blockchain history, decompress to build the summary preserving zero trust
  • Rescan that checks all the hashes to detect and recover from on disk corruption(0.8)
  • Backwards header sync, then fill blocks, reverse header-fetching sync
  • Start as a SPV (lite) client, validate to become a full node in the background

Mining

  • P2Pool
    • If P2Pools have a significant amount of hashpower, and their shares carry commitments to transactions, P2Pool can create sub-minute evidence of intent to mine — coupled with double-spend-alerts can create pretty good fast confidence.
  • Simplistic GPU miner (bitcoin screensaver?)
  • Smarter tree rebulding
  • Share in lieu of TXN fees
    • No funds for anti-dos fee? Generate at least a diff N share with a commitment to the txn in the coinbase (paying whomever), distribute along with the txn (allows us to keep mining as our singular POW system, but still use small amounts of it without a pooling solution that works at cpu miner levels!)
    • Directly pay pools txn fees via computation, attempting blocks that pay them.
  • Directed graph fee/priority computation (a later txn can pay fee for parents, must avoid memory DOS)

Fee tweaks

  • Consider net unspent output change (txn which decrease the unspent outputs on the network should have fewer fees) (Done, not quite net change as that has some weird incentives, but some of the signature size is ignored for priority calculating purposes now)
  • NAK for dropped transactions so clients (sometimes) know when their txn aren't making progress

Proofs

If full nodes become expensive to operate in the future then they may become uncommon and this could compromise the security of Bitcoin. This risk can be reduced if it's made possible for Bitcoin nodes to check all the rules at random and transmit compact proofs of rule violations. If this is done even if there is only one honest full node in the world the system is secure so long as it can communicate to all others.

In general, in any deterministic computation process if you have simple state updates and commit to the sequence of states a compact proof of invalidity can be generated by producing a hash tree fragment to the first invalidate state transition.

Ideas in this space have been previously discussed under the banner of proof-of-treachery [1].

Right now not all of the rules can be checked randomly or have compact proofs.

  • SPV header checks — time, target, difficulty, already have them. But if future SPV nodes don't really check all the headers in the future it may be useful to arrange old header times in a merkle mountain range to proofs of sum difficulty and compact proofs of incorrect difficulty.
  • Proof of invalid script
    • Possible in the current system: Proof is tree fragments for the invalid txn in question as well as one invalid input (no need to include more than one).
    • Could be made more efficient by including commitments to intermediate states, but with the 201 opcode limit all scripts are compact to verify in Bitcoin without doing anything fancier.
    • Also proves nlocktime, etc.
    • To prove output value greater than inputs all inputs must be provided in the proof.
  • Proof of double spend
    • Possible in the current system: Proof is tree fragments for the two transactions which spend the same input.
  • Proof of false inflation
    • Not possible without more data: The coinbase payment is the sum of fees in a block and the subsidy. Fees require knowing the transaction's inputs output values, to check subsidy you must not only have all the transactions but all their inputs as well.
    • This is fixable. Replace the hash tree with one that H((fees_left+fees_right)||H(left)||H(right)) so that the tree commits to the fees. Nodes can randomly check this by grabbing a random txn and checking its inputs, and compactly prove violation by showing where the fees don't match their commitments.
  • Proof of block too large
    • Similar to false inflation, requires all the transactions, can similarly be solved by including the sum of txn sizes in the tree.
  • Proof of spending a non-existing input
    • Requires additional data: proof requires scanning the entire blockchain history
    • Requires a UXTO set commitment as well as a tree for outputs created and consumed within one block. Proof is a pair of tree fragments for the higher and lower records for the missing entry, and another pair for the outputs created within a block but consumed.