Difference between revisions of "BIP 0322"

From Bitcoin Wiki
Jump to: navigation, search
(Update BIP text with latest version from https://github.com/bitcoin/bips/blob/b5723035e23896d0/bip-0322.mediawiki)
 
(Update BIP text with latest version from https://github.com/bitcoin/bips/blob/c134a853a9fc0657/bip-0322.mediawiki)
 
Line 37: Line 37:
  
 
A new structure <code>SignatureProof</code> is added, which is a simple serializable scriptSig & witness container.
 
A new structure <code>SignatureProof</code> is added, which is a simple serializable scriptSig & witness container.
 
Two actions "Sign" and "Verify" are defined along with one ''purpose'', "SignMessage", with the ability to expand in the future to add a potential "ProveFunds" purpose.
 
  
 
=== SignatureProof container ===
 
=== SignatureProof container ===
 
{|class="wikitable" style="text-align: center;"
 
|-
 
!Type
 
!Length
 
!Name
 
!Comment
 
|-
 
|Uint32||4||version||BIP322 version format; must be equal to 1; if > 1, verifier must abort the verification process
 
|-
 
|Uint8||1||entries||number of proof entries<ref><strong>Why support multiple proofs?</strong> It is non-trivial to check a large number of individual proofs for duplicates. Software could be written to do so, but it seems more efficient to build this check into the specification itself.</ref>
 
|}
 
 
The above is followed by [entries] number of signature entries:
 
  
 
{|class="wikitable" style="text-align: center;"
 
{|class="wikitable" style="text-align: center;"
Line 83: Line 67:
 
!Description
 
!Description
 
|-
 
|-
|INCOMPLETE||One or several of the given challenges had an empty proof. The prover may need some other entity to complete the proof.
+
|INCOMPLETE||Empty proof.
 
|-
 
|-
|INCONCLUSIVE||One or several of the given proofs was consensus-valid but policy-invalid.
+
|INCONCLUSIVE||The given proof was consensus-valid but policy-invalid.
 
|-
 
|-
|VALID||All proofs were deemed valid.
+
|VALID||The proof was valid.
 
|-
 
|-
|INVALID||One or more of the given proofs were invalid
+
|INVALID||The proof was invalid
 
|-
 
|-
 
|ERROR||An error was encountered
 
|ERROR||An error was encountered
Line 96: Line 80:
 
== Signing and Verifying ==
 
== Signing and Verifying ==
  
If the challenge consists of a single address and the address is in the P2PKH (legacy) format, sign using the legacy format (further information below). Otherwise continue as stated below.
+
If the challenge consists of an address is in the P2PKH (legacy) format, sign using the legacy format (further information below). Otherwise continue as stated below.
 
 
Let there be an empty set <code>inputs</code> which is populated and tested at each call to one of the actions below.
 
  
=== Purpose: SignMessage ===
+
For both cases, generate a sighash based on the given scriptPubKey and message as follows:
  
The "SignMessage" purpose generates a sighash based on a scriptPubKey and a message. It emits a VALID verification result code unless otherwise stated.
 
 
# Return INVALID if scriptPubKey already exists in <code>inputs</code> set, otherwise insert it<ref><strong>Why track duplicates?</strong> Because a 3-entry proof is not proving 3 entries unless they are all distinct</ref>
 
 
# Define the message pre-image as the sequence "Bitcoin Signed Message:\n" concatenated with the message, encoded in UTF-8 using Normalization Form Compatibility Decomposition (NFKD)
 
# Define the message pre-image as the sequence "Bitcoin Signed Message:\n" concatenated with the message, encoded in UTF-8 using Normalization Form Compatibility Decomposition (NFKD)
 
# Let sighash = sha256(sha256(scriptPubKey || pre-image))
 
# Let sighash = sha256(sha256(scriptPubKey || pre-image))
Line 110: Line 89:
 
A private key may be used directly to sign a message. In this case, its P2WPKH bech32 address shall be derived, and used as the input.
 
A private key may be used directly to sign a message. In this case, its P2WPKH bech32 address shall be derived, and used as the input.
  
=== Action: Sign ===
+
=== Signing ===
  
The "Sign" action takes as input a purpose. It returns a signature or fails.
+
The signature is generated as follows:
  
# Obtain the sighash and scriptPubKey from the purpose; FAIL if not VALID
 
 
# Derive the private key privkey for the scriptPubKey; FAIL if not VALID
 
# Derive the private key privkey for the scriptPubKey; FAIL if not VALID
 
# Generate and return a signature sig with privkey=privkey, sighash=sighash
 
# Generate and return a signature sig with privkey=privkey, sighash=sighash
  
The resulting signature proof should be encoded using base64 encoding.
+
=== Verifying ===
  
=== Action: Verify ===
+
Verify a proof, given a standard flags value, a script sig, an optional witness, and a derived sighash as described above.
 
 
The "Verify" action takes as input a standard flags value, a script sig, an optional witness, and a purpose.
 
It emits one of INCONCLUSIVE, VALID, INVALID, or ERROR.
 
  
 
While omitted below, ERROR is returned if an unforeseen error occurs at any point in the process. A concrete example of this is if a legacy proof is given as input to a non-legacy address; the deserialization of the proof will fail in this case, and this should result in an ERROR result.
 
While omitted below, ERROR is returned if an unforeseen error occurs at any point in the process. A concrete example of this is if a legacy proof is given as input to a non-legacy address; the deserialization of the proof will fail in this case, and this should result in an ERROR result.
  
# Obtain the sighash and scriptPubKey from the purpose; pass on result code if not VALID
 
 
# Verify Script with flags=consensus flags (currently P2SH, DERSIG, NULLDUMMY, CLTV, CSV, WITNESS), scriptSig=script sig, scriptPubKey=scriptPubKey, witness=witness, and sighash=sighash
 
# Verify Script with flags=consensus flags (currently P2SH, DERSIG, NULLDUMMY, CLTV, CSV, WITNESS), scriptSig=script sig, scriptPubKey=scriptPubKey, witness=witness, and sighash=sighash
 
# Return INVALID if verification fails
 
# Return INVALID if verification fails
 
# Verify Script with flags=standard flags (above plus STRICTENC, MINIMALDATA, etc.), scriptSig=script sig, scriptPubKey=scriptPubKey, witness=witness, and sighash=sighash
 
# Verify Script with flags=standard flags (above plus STRICTENC, MINIMALDATA, etc.), scriptSig=script sig, scriptPubKey=scriptPubKey, witness=witness, and sighash=sighash
 
# Return VALID if verification succeeds, otherwise return INCONCLUSIVE
 
# Return VALID if verification succeeds, otherwise return INCONCLUSIVE
 
=== Multiple Proofs ===
 
 
When more than one proof is created or verified, repeat the operation for each proof, retaining the inputs set. As noted, if the same input appears more than once, the operation must fail accordingly.
 
 
Note that the order of the entries in the proof must match the order of the entries given by the verifier.
 
 
* If any of the proofs are empty during a verification process, skip the verification and set the INCOMPLETE flag
 
* If a verification call returns ERROR or INVALID, return ERROR or INVALID immediately, ignoring as yet unverified entries
 
* After all verifications complete,
 
** return INCONCLUSIVE if any verification call returned INCONCLUSIVE
 
** return INCOMPLETE if the INCOMPLETE flag is set
 
** return VALID
 
  
 
== Legacy format ==
 
== Legacy format ==
  
The legacy format is restricted to the legacy P2PKH address format, and restricted to one single challenge (address).
+
The legacy format is restricted to the legacy P2PKH address format.
  
Any other input (e.g. multiple addresses, or non-P2PKH address format(s)) must be signed using the new format described above.
+
Any other input (i.e. non-P2PKH address format) must be signed using the new format described above.
  
 
=== Signing ===
 
=== Signing ===
Line 177: Line 138:
  
 
This specification is backwards compatible with the legacy signmessage/verifymessage specification through the special case as described above.
 
This specification is backwards compatible with the legacy signmessage/verifymessage specification through the special case as described above.
 
== Rationale ==
 
 
<references/>
 
  
 
== Reference implementation ==
 
== Reference implementation ==
Line 231: Line 188:
  
 
== Test vectors ==
 
== Test vectors ==
 +
 +
(TODO: update test vectors, which are based on previous iteration where signature proofs contained additional data)
  
 
== Native segwit test vector ==
 
== Native segwit test vector ==

Latest revision as of 07:36, 2 August 2020

This page describes a BIP (Bitcoin Improvement Proposal).
Please see BIP 2 for more information about BIPs and creating them. Please do not just create a wiki page.

Please do not modify this page. This is a mirror of the BIP from the source Git repository here.

  BIP: 322
  Layer: Applications
  Title: Generic Signed Message Format
  Author: Karl-Johan Alm <karljohan-alm@garage.co.jp>
  Comments-Summary: No comments yet.
  Comments-URI: https://github.com/bitcoin/bips/wiki/Comments:BIP-0322
  Status: Draft
  Type: Standards Track
  Created: 2018-09-10
  License: CC0-1.0

Abstract

A standard for interoperable generic signed messages based on the Bitcoin Script format.

Background

  • Assume two actors, a prover P and a verifier V.
  • P wants to prove that they own the private key k associated with a given address A (which in turn is derived from the pubkey kG).
  • Let V generate a message M and hand this to P.
  • P generates a signature S by signing the message M using k. Given S, V can prove that P has the private key associated with A.

The astute reader will notice that the above is missing a critical part, namely the pubkey kG, without which the verifier cannot actually verify the message. The current message signing standard solves this via a cryptographic trick, wherein the signature S above is a special "recoverable signature" type. Given the message M and the signature S, it is then possible to recover the pubkey kG. The system thus derives the address for the pubkey kG, and if it does not match A, the proof is deemed invalid.

While this is a neat trick, it unnecessarily restricts and complicates the message signing mechanism; for instance, it is currently not possible to sign a message for a P2SH address, because there is no pubkey to recover from the resulting signature.

Motivation

The current message signing standard only works for P2PKH (1...) addresses. By extending it to use a Bitcoin Script based approach, it could be made more generic without causing a too big burden on implementers, who most likely have access to Bitcoin Script interpreters already.

Specification

A new structure SignatureProof is added, which is a simple serializable scriptSig & witness container.

SignatureProof container

Type Length Name Comment
VarInt 1-8 scriptsiglen Number of bytes in scriptSig data
Uint8* [scriptsiglen] scriptsig ScriptSig data
VarInt 1-8 witlen Number of entries in witness stack
Uint8[]* [witlen] wit Witness stack, as [witlen] uint8* vectors, each one prepended with a varint of its size

In some cases, the scriptsig or wit may be empty. If both are empty, the proof is incomplete.

Result Codes

A verification call will return a result code according to the table below.

Code Description
INCOMPLETE Empty proof.
INCONCLUSIVE The given proof was consensus-valid but policy-invalid.
VALID The proof was valid.
INVALID The proof was invalid
ERROR An error was encountered

Signing and Verifying

If the challenge consists of an address is in the P2PKH (legacy) format, sign using the legacy format (further information below). Otherwise continue as stated below.

For both cases, generate a sighash based on the given scriptPubKey and message as follows:

  1. Define the message pre-image as the sequence "Bitcoin Signed Message:\n" concatenated with the message, encoded in UTF-8 using Normalization Form Compatibility Decomposition (NFKD)
  2. Let sighash = sha256(sha256(scriptPubKey || pre-image))

A private key may be used directly to sign a message. In this case, its P2WPKH bech32 address shall be derived, and used as the input.

Signing

The signature is generated as follows:

  1. Derive the private key privkey for the scriptPubKey; FAIL if not VALID
  2. Generate and return a signature sig with privkey=privkey, sighash=sighash

Verifying

Verify a proof, given a standard flags value, a script sig, an optional witness, and a derived sighash as described above.

While omitted below, ERROR is returned if an unforeseen error occurs at any point in the process. A concrete example of this is if a legacy proof is given as input to a non-legacy address; the deserialization of the proof will fail in this case, and this should result in an ERROR result.

  1. Verify Script with flags=consensus flags (currently P2SH, DERSIG, NULLDUMMY, CLTV, CSV, WITNESS), scriptSig=script sig, scriptPubKey=scriptPubKey, witness=witness, and sighash=sighash
  2. Return INVALID if verification fails
  3. Verify Script with flags=standard flags (above plus STRICTENC, MINIMALDATA, etc.), scriptSig=script sig, scriptPubKey=scriptPubKey, witness=witness, and sighash=sighash
  4. Return VALID if verification succeeds, otherwise return INCONCLUSIVE

Legacy format

The legacy format is restricted to the legacy P2PKH address format.

Any other input (i.e. non-P2PKH address format) must be signed using the new format described above.

Signing

Given the P2PKH address a and the message m, and the pubkey-hash function pkh(P) = ripemd160(sha256(P)):

  1. let p be the pubkey-hash pkh(P) for the pubkey P, contained in a
  2. let x be the private key associated with P so that pkh(xG) = p
  3. let digest be SHA56d("Bitcoin Signed Message:\n"||m)
  4. create a compact signature sig (aka "recoverable ECDSA signature") using x on digest

The resulting proof is sig, serialized using the base64 encoding.

Verifying

Given the P2PKH address a, the message m, the compact signature sig, and the pubkey-hash function pkh(P) = ripemd160(sha256(P)):

  1. let p be the pubkey-hash pkh(P) for the pubkey P, contained in a
  2. let digest be SHA56d("Bitcoin Signed Message:\n"||m)
  3. attempt pubkey recovery for digest using the signature sig and store the resulting pubkey into Q
    1. fail verification if pubkey recovery above fails
  4. let q be the pubkey-hash pkh(Q) for the pubkey Q
  5. if p == q, the proof is valid, otherwise it is invalid

Compatibility

This specification is backwards compatible with the legacy signmessage/verifymessage specification through the special case as described above.

Reference implementation

  1. Pull request to Bitcoin Core: https://github.com/bitcoin/bitcoin/pull/16440

Acknowledgements

Thanks to David Harding, Jim Posen, Kalle Rosenbaum, Pieter Wuille, and many others for their feedback on the specification.

References

  1. Original mailing list thread: https://lists.linuxfoundation.org/pipermail/bitcoin-dev/2018-March/015818.html

Copyright

This document is licensed under the Creative Commons CC0 1.0 Universal license.

Consensus and standard flags

Each flag is associated with some type of enforced rule (most often a soft fork). There are two sets of flags: consensus flags (which result in a block being rejected, if violated), and policy flags (which result in a transaction being accepted only if it is contained within an actual block, and rejected otherwise, if violated). The policy flags are a super-set of the consensus flags.

BIP322 specifies that a proof that validates for both rulesets is valid, a proof that validates for consensus rules, but not for policy rules, is "inconclusive", and a proof that does not validate for consensus rules is "invalid" (regardless of policy rule validation).

The ruleset sometimes changes. This BIP does not intend to be complete, nor does it indicate enforcement of rules, it simply lists the rules as they stand at the point of writing.

Consensus rules

  • P2SH: evaluate P2SH (BIP16) subscripts
  • DERSIG: enforce strict DER (BIP66) compliance
  • NULLDUMMY: enforce NULLDUMMY (BIP147)
  • CHECKLOCKTIMEVERIFY: enable CHECKLOCKTIMEVERIFY (BIP65)
  • CHECKSEQUENCEVERIFY: enable CHECKSEQUENCEVERIFY (BIP112)
  • WITNESS: enable WITNESS (BIP141)

Policy rules

All of the above, plus (subject to change):

  • STRICTENC: non-strict DER signature or undefined hashtype
  • MINIMALDATA: require minimal encodings for all push operations
  • DISCOURAGE_UPGRADABLE_NOPS: discourage use of NOPs reserved for upgrades
  • CLEANSTACK: require that only a single stack element remains after evaluation
  • MINIMALIF: Segwit script only: require the argument of OP_IF/NOTIF to be exactly 0x01 or empty vector
  • NULLFAIL: signature(s) must be empty vector if a CHECK(MULTI)SIG operation failed
  • LOW_S: signature with S > order/2 in a checksig operation
  • DISCOURAGE_UPGRADABLE_WITNESS_PROGRAM: v1-16 witness programs are non-standard (i.e. forbidden)
  • WITNESS_PUBKEYTYPE: public keys in segregated witness scripts must be compressed
  • CONST_SCRIPTCODE: OP_CODESEPARATOR and FindAndDelete fail any non-segwit scripts

Test vectors

(TODO: update test vectors, which are based on previous iteration where signature proofs contained additional data)

Native segwit test vector

address      = bcrt1qe7nte4zk4ayly5tc53dtdjupgkz0lr8azx3rzz
scriptpubkey = 0014cfa6bcd456af49f25178a45ab6cb814584ff8cfd
message      = hello
preimage     = 0014cfa6bcd456af49f25178a45ab6cb814584ff8cfd426974636f696e205369
               676e6564204d6573736167653a0a68656c6c6f
               (scriptpubkey || "Bitcoin Signed Message:\nhello")
sighash      = 790eef86c204f0bff969ff822121317aa34eff0215dbd30ccf031e7b2f3f0cc1
               (sha256d(preimage), displayed in big-endian)

The proof becomes:

HEX:    01000000010002473044022075b4fb40421d55c55462879cb352a85eeb3af2138d3f0290
        2c9143f12870f5f70220119c2995c1661138142f3899c1fd6d1af7e790e0e081be72db9c
        e7bf5b5b932901210290beccd02b73eca57467b2b6f1e47161a9b76a5e67586e7c1dee9e
        a6e2dcd869

Base64: AQAAAAEAAkcwRAIgdbT7QEIdVcVUYoecs1KoXus68hONPwKQLJFD8Shw9fcCIBGcKZXBZhE4
        FC84mcH9bRr355Dg4IG+ctuc579bW5MpASECkL7M0Ctz7KV0Z7K28eRxYam3al5nWG58He6e
        puLc2Gk=

Split into components:

Type Length Name Value Comment
Uint32 4 flags 01000000 proof format version
Uint8 1 entries 01 1 entry
VarInt 1-8 scriptsiglen 00 0 byte scriptsig
VarInt 1-8 wit entries 02 2 witness stack entries
VarInt 1-8 entry1len 47 71 byte entry
Uint8[71] 71 entry1 3044022075b4fb40421d55c55462879cb352a85eeb3af213

8d3f02902c9143f12870f5f70220119c2995c1661138142f 3899c1fd6d1af7e790e0e081be72db9ce7bf5b5b932901||Witness stack item 1

VarInt 1-8 entry2len 21 33 byte entry
Uint8[33] 33 entry2 0290beccd02b73eca57467b2b6f1e47161a9b76a5e67586e

7c1dee9ea6e2dcd869||Witness stack item 2

The above test vector is for a bech32 P2WPKH (native segwit) address. (Once BIP solidifies, will add test vector for other types.)