protocol.rst
1 Protocol specification 2 ====================== 3 4 .. warning:: All objects sent on the network should support protocol v3 5 starting on Sun, 16 Nov 2014 22:00:00 GMT. 6 7 .. toctree:: 8 :maxdepth: 2 9 10 Common standards 11 ---------------- 12 13 Hashes 14 ^^^^^^ 15 16 Most of the time `SHA-512 <http://en.wikipedia.org/wiki/SHA-2>`_ hashes are 17 used, however `RIPEMD-160 <http://en.wikipedia.org/wiki/RIPEMD>`_ is also used 18 when creating an address. 19 20 A double-round of SHA-512 is used for the Proof Of Work. Example of 21 double-SHA-512 encoding of string "hello": 22 23 .. highlight:: nasm 24 25 :: 26 27 hello 28 9b71d224bd62f3785d96d46ad3ea3d73319bfbc2890caadae2dff72519673ca72323c3d99ba5c11d7c7acc6e14b8c5da0c4663475c2e5c3adef46f73bcdec043(first round of sha-512) 29 0592a10584ffabf96539f3d780d776828c67da1ab5b169e9e8aed838aaecc9ed36d49ff1423c55f019e050c66c6324f53588be88894fef4dcffdb74b98e2b200(second round of sha-512) 30 31 For Bitmessage addresses (RIPEMD-160) this would give: 32 33 :: 34 35 hello 36 9b71d224bd62f3785d96d46ad3ea3d73319bfbc2890caadae2dff72519673ca72323c3d99ba5c11d7c7acc6e14b8c5da0c4663475c2e5c3adef46f73bcdec043(first round is sha-512) 37 79a324faeebcbf9849f310545ed531556882487e (with ripemd-160) 38 39 40 Common structures 41 ----------------- 42 43 All integers are encoded in big endian. (This is different from Bitcoin). 44 45 .. list-table:: Message structure 46 :header-rows: 1 47 :widths: auto 48 49 * - Field Size 50 - Description 51 - Data type 52 - Comments 53 * - 4 54 - magic 55 - uint32_t 56 - Magic value indicating message origin network, and used to seek to next 57 message when stream state is unknown 58 * - 12 59 - command 60 - char[12] 61 - ASCII string identifying the packet content, NULL padded (non-NULL 62 padding results in packet rejected) 63 * - 4 64 - length 65 - uint32_t 66 - Length of payload in number of bytes. Because of other restrictions, 67 there is no reason why this length would ever be larger than 1600003 68 bytes. Some clients include a sanity-check to avoid processing messages 69 which are larger than this. 70 * - 4 71 - checksum 72 - uint32_t 73 - First 4 bytes of sha512(payload) 74 * - ? 75 - message_payload 76 - uchar[] 77 - The actual data, a :ref:`message <msg-types>` or an object_. 78 Not to be confused with objectPayload. 79 80 Known magic values: 81 82 +-------------+-------------------+ 83 | Magic value | Sent over wire as | 84 +=============+===================+ 85 | 0xE9BEB4D9 | E9 BE B4 D9 | 86 +-------------+-------------------+ 87 88 .. _varint: 89 90 Variable length integer 91 ^^^^^^^^^^^^^^^^^^^^^^^ 92 93 Integer can be encoded depending on the represented value to save space. 94 Variable length integers always precede an array/vector of a type of data that 95 may vary in length. Varints **must** use the minimum possible number of bytes to 96 encode a value. For example, the value 6 can be encoded with one byte therefore 97 a varint that uses three bytes to encode the value 6 is malformed and the 98 decoding task must be aborted. 99 100 +---------------+----------------+------------------------------------------+ 101 | Value | Storage length | Format | 102 +===============+================+==========================================+ 103 | < 0xfd | 1 | uint8_t | 104 +---------------+----------------+------------------------------------------+ 105 | <= 0xffff | 3 | 0xfd followed by the integer as uint16_t | 106 +---------------+----------------+------------------------------------------+ 107 | <= 0xffffffff | 5 | 0xfe followed by the integer as uint32_t | 108 +---------------+----------------+------------------------------------------+ 109 | - | 9 | 0xff followed by the integer as uint64_t | 110 +---------------+----------------+------------------------------------------+ 111 112 Variable length string 113 ^^^^^^^^^^^^^^^^^^^^^^ 114 115 Variable length string can be stored using a variable length integer followed by 116 the string itself. 117 118 +------------+-------------+------------+----------------------------------+ 119 | Field Size | Description | Data type | Comments | 120 +============+=============+============+==================================+ 121 | 1+ | length | |var_int| | Length of the string | 122 +------------+-------------+------------+----------------------------------+ 123 | ? | string | char[] | The string itself (can be empty) | 124 +------------+-------------+------------+----------------------------------+ 125 126 Variable length list of integers 127 ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ 128 129 n integers can be stored using n+1 :ref:`variable length integers <varint>` 130 where the first var_int equals n. 131 132 +------------+-------------+-----------+----------------------------+ 133 | Field Size | Description | Data type | Comments | 134 +============+=============+===========+============================+ 135 | 1+ | count | |var_int| | Number of var_ints below | 136 +------------+-------------+-----------+----------------------------+ 137 | 1+ | | var_int | The first value stored | 138 +------------+-------------+-----------+----------------------------+ 139 | 1+ | | var_int | The second value stored... | 140 +------------+-------------+-----------+----------------------------+ 141 | 1+ | | var_int | etc... | 142 +------------+-------------+-----------+----------------------------+ 143 144 .. |var_int| replace:: :ref:`var_int <varint>` 145 146 Network address 147 ^^^^^^^^^^^^^^^ 148 149 When a network address is needed somewhere, this structure is used. Network 150 addresses are not prefixed with a timestamp or stream in the version_ message. 151 152 .. list-table:: 153 :header-rows: 1 154 :widths: auto 155 156 * - Field Size 157 - Description 158 - Data type 159 - Comments 160 * - 8 161 - time 162 - uint64 163 - the Time. 164 * - 4 165 - stream 166 - uint32 167 - Stream number for this node 168 * - 8 169 - services 170 - uint64_t 171 - same service(s) listed in version_ 172 * - 16 173 - IPv6/4 174 - char[16] 175 - IPv6 address. IPv4 addresses are written into the message as a 16 byte 176 `IPv4-mapped IPv6 address <http://en.wikipedia.org/wiki/IPv6#IPv4-mapped_IPv6_addresses>`_ 177 (12 bytes 00 00 00 00 00 00 00 00 00 00 FF FF, followed by the 4 bytes of 178 the IPv4 address). 179 * - 2 180 - port 181 - uint16_t 182 - port number 183 184 Inventory Vectors 185 ^^^^^^^^^^^^^^^^^ 186 187 Inventory vectors are used for notifying other nodes about objects they have or 188 data which is being requested. Two rounds of SHA-512 are used, resulting in a 189 64 byte hash. Only the first 32 bytes are used; the later 32 bytes are ignored. 190 191 Inventory vectors consist of the following data format: 192 193 +------------+-------------+-----------+--------------------+ 194 | Field Size | Description | Data type | Comments | 195 +============+=============+===========+====================+ 196 | 32 | hash | char[32] | Hash of the object | 197 +------------+-------------+-----------+--------------------+ 198 199 Encrypted payload 200 ^^^^^^^^^^^^^^^^^ 201 202 Bitmessage uses `ECIES <https://en.wikipedia.org/wiki/Integrated_Encryption_Scheme>`_ to encrypt its messages. For more information see :doc:`encryption` 203 204 .. include:: encrypted_payload.rst 205 206 Unencrypted Message Data 207 ^^^^^^^^^^^^^^^^^^^^^^^^ 208 209 .. list-table:: 210 :header-rows: 1 211 :widths: auto 212 213 * - Field Size 214 - Description 215 - Data type 216 - Comments 217 * - 1+ 218 - msg_version 219 - var_int 220 - Message format version. **This field is not included after the 221 protocol v3 upgrade period**. 222 * - 1+ 223 - address_version 224 - var_int 225 - Sender's address version number. This is needed in order to calculate 226 the sender's address to show in the UI, and also to allow for forwards 227 compatible changes to the public-key data included below. 228 * - 1+ 229 - stream 230 - var_int 231 - Sender's stream number 232 * - 4 233 - behavior bitfield 234 - uint32_t 235 - A bitfield of optional behaviors and features that can be expected from 236 the node with this pubkey included in this msg message (the sender's 237 pubkey). 238 * - 64 239 - public signing key 240 - uchar[] 241 - The ECC public key used for signing (uncompressed format; 242 normally prepended with \x04) 243 * - 64 244 - public encryption key 245 - uchar[] 246 - The ECC public key used for encryption (uncompressed format; 247 normally prepended with \x04) 248 * - 1+ 249 - nonce_trials_per_byte 250 - var_int 251 - Used to calculate the difficulty target of messages accepted by this 252 node. The higher this value, the more difficult the Proof of Work must 253 be before this individual will accept the message. This number is the 254 average number of nonce trials a node will have to perform to meet the 255 Proof of Work requirement. 1000 is the network minimum so any lower 256 values will be automatically raised to 1000. **This field is new and is 257 only included when the address_version >= 3**. 258 * - 1+ 259 - extra_bytes 260 - var_int 261 - Used to calculate the difficulty target of messages accepted by this 262 node. The higher this value, the more difficult the Proof of Work must 263 be before this individual will accept the message. This number is added 264 to the data length to make sending small messages more difficult. 265 1000 is the network minimum so any lower values will be automatically 266 raised to 1000. **This field is new and is only included when the 267 address_version >= 3**. 268 * - 20 269 - destination ripe 270 - uchar[] 271 - The ripe hash of the public key of the receiver of the message 272 * - 1+ 273 - encoding 274 - var_int 275 - :ref:`Message Encoding <msg-encodings>` type 276 * - 1+ 277 - message_length 278 - var_int 279 - Message Length 280 * - message_length 281 - message 282 - uchar[] 283 - The message. 284 * - 1+ 285 - ack_length 286 - var_int 287 - Length of the acknowledgement data 288 * - ack_length 289 - ack_data 290 - uchar[] 291 - The acknowledgement data to be transmitted. This takes the form of a 292 Bitmessage protocol message, like another msg message. The POW therein 293 must already be completed. 294 * - 1+ 295 - sig_length 296 - var_int 297 - Length of the signature 298 * - sig_length 299 - signature 300 - uchar[] 301 - The ECDSA signature which covers the object header starting with the 302 time, appended with the data described in this table down to the 303 ack_data. 304 305 .. _msg-encodings: 306 307 Message Encodings 308 """"""""""""""""" 309 310 .. list-table:: 311 :header-rows: 1 312 :widths: auto 313 314 * - Value 315 - Name 316 - Description 317 * - 0 318 - IGNORE 319 - Any data with this number may be ignored. The sending node might simply 320 be sharing its public key with you. 321 * - 1 322 - TRIVIAL 323 - UTF-8. No 'Subject' or 'Body' sections. Useful for simple strings 324 of data, like URIs or magnet links. 325 * - 2 326 - SIMPLE 327 - UTF-8. Uses 'Subject' and 'Body' sections. No MIME is used. 328 :: 329 messageToTransmit = 'Subject:' + subject + '\n' + 'Body:' + message 330 * - 3 331 - EXTENDED 332 - See :doc:`extended_encoding` 333 334 Further values for the message encodings can be decided upon by the community. 335 Any MIME or MIME-like encoding format, should they be used, should make use of 336 Bitmessage's 8-bit bytes. 337 338 .. _behavior-bitfield: 339 340 Pubkey bitfield features 341 """""""""""""""""""""""" 342 343 .. list-table:: 344 :header-rows: 1 345 :widths: auto 346 347 * - Bit 348 - Name 349 - Description 350 * - 0 351 - undefined 352 - The most significant bit at the beginning of the structure. Undefined 353 * - 1 354 - undefined 355 - The next most significant bit. Undefined 356 * - ... 357 - ... 358 - ... 359 * - 27 360 - onion_router 361 - (**Proposal**) Node can be used to onion-route messages. In theory any 362 node can onion route, but since it requires more resources, they may have 363 the functionality disabled. This field will be used to indicate that the 364 node is willing to do this. 365 * - 28 366 - forward_secrecy 367 - (**Proposal**) Receiving node supports a forward secrecy encryption 368 extension. The exact design is pending. 369 * - 29 370 - chat 371 - (**Proposal**) Address if for chatting rather than messaging. 372 * - 30 373 - include_destination 374 - (**Proposal**) Receiving node expects that the RIPE hash encoded in their 375 address preceedes the encrypted message data of msg messages bound for 376 them. 377 378 .. note:: since hardly anyone implements this, this will be redesigned as 379 `simple recipient verification <https://github.com/Bitmessage/PyBitmessage/pull/808#issuecomment-170189856>`_ 380 * - 31 381 - does_ack 382 - If true, the receiving node does send acknowledgements (rather than 383 dropping them). 384 385 .. _msg-types: 386 387 Message types 388 ------------- 389 390 Undefined messages received on the wire must be ignored. 391 392 version 393 ^^^^^^^ 394 395 When a node creates an outgoing connection, it will immediately advertise its 396 version. The remote node will respond with its version. No futher communication 397 is possible until both peers have exchanged their version. 398 399 .. list-table:: Payload 400 :header-rows: 1 401 :widths: auto 402 403 * - Field Size 404 - Description 405 - Data type 406 - Comments 407 * - 4 408 - version 409 - int32_t 410 - Identifies protocol version being used by the node. Should equal 3. 411 Nodes should disconnect if the remote node's version is lower but 412 continue with the connection if it is higher. 413 * - 8 414 - services 415 - uint64_t 416 - bitfield of features to be enabled for this connection 417 * - 8 418 - timestamp 419 - int64_t 420 - standard UNIX timestamp in seconds 421 * - 26 422 - addr_recv 423 - net_addr 424 - The network address of the node receiving this message (not including the 425 time or stream number) 426 * - 26 427 - addr_from 428 - net_addr 429 - The network address of the node emitting this message (not including the 430 time or stream number and the ip itself is ignored by the receiver) 431 * - 8 432 - nonce 433 - uint64_t 434 - Random nonce used to detect connections to self. 435 * - 1+ 436 - user_agent 437 - var_str 438 - :doc:`useragent` (0x00 if string is 0 bytes long). Sending nodes must not 439 include a user_agent longer than 5000 bytes. 440 * - 1+ 441 - stream_numbers 442 - var_int_list 443 - The stream numbers that the emitting node is interested in. Sending nodes 444 must not include more than 160000 stream numbers. 445 446 A "verack" packet shall be sent if the version packet was accepted. Once you 447 have sent and received a verack messages with the remote node, send an addr 448 message advertising up to 1000 peers of which you are aware, and one or more 449 inv messages advertising all of the valid objects of which you are aware. 450 451 .. list-table:: The following services are currently assigned 452 :header-rows: 1 453 :widths: auto 454 455 * - Value 456 - Name 457 - Description 458 * - 1 459 - NODE_NETWORK 460 - This is a normal network node. 461 * - 2 462 - NODE_SSL 463 - This node supports SSL/TLS in the current connect (python < 2.7.9 only 464 supports a SSL client, so in that case it would only have this on when 465 the connection is a client). 466 * - 3 467 - NODE_POW 468 - (**Proposal**) This node may do PoW on behalf of some its peers (PoW 469 offloading/delegating), but it doesn't have to. Clients may have to meet 470 additional requirements (e.g. TLS authentication) 471 * - 4 472 - NODE_DANDELION 473 - Node supports `dandelion <https://github.com/gfanti/bips/blob/master/bip-dandelion.mediawiki>`_ 474 475 verack 476 ^^^^^^ 477 478 The *verack* message is sent in reply to *version*. This message consists of 479 only a :ref:`message header <Message structure>` with the command string 480 "verack". The TCP timeout starts out at 20 seconds; after verack messages are 481 exchanged, the timeout is raised to 10 minutes. 482 483 If both sides announce that they support SSL, they **must** perform an SSL 484 handshake immediately after they both send and receive verack. During this SSL 485 handshake, the TCP client acts as an SSL client, and the TCP server acts as an 486 SSL server. The current implementation (v0.5.4 or later) requires the 487 AECDH-AES256-SHA cipher over TLSv1 protocol, and prefers the secp256k1 curve 488 (but other curves may be accepted, depending on the version of python and 489 OpenSSL used). 490 491 addr 492 ^^^^ 493 494 Provide information on known nodes of the network. Non-advertised nodes should 495 be forgotten after typically 3 hours 496 497 Payload: 498 499 +------------+-------------+-----------+---------------------------------------+ 500 | Field Size | Description | Data type | Comments | 501 +============+=============+===========+=======================================+ 502 | 1+ | count | |var_int| | Number of address entries (max: 1000) | 503 +------------+-------------+-----------+---------------------------------------+ 504 | 38 | addr_list | net_addr | Address of other nodes on the network.| 505 +------------+-------------+-----------+---------------------------------------+ 506 507 inv 508 ^^^ 509 510 Allows a node to advertise its knowledge of one or more objects. Payload 511 (maximum payload length: 50000 items): 512 513 +------------+-------------+------------+-----------------------------+ 514 | Field Size | Description | Data type | Comments | 515 +============+=============+============+=============================+ 516 | ? | count | |var_int| | Number of inventory entries | 517 +------------+-------------+------------+-----------------------------+ 518 | 32x? | inventory | inv_vect[] | Inventory vectors | 519 +------------+-------------+------------+-----------------------------+ 520 521 getdata 522 ^^^^^^^ 523 524 getdata is used in response to an inv message to retrieve the content of a 525 specific object after filtering known elements. 526 527 Payload (maximum payload length: 50000 entries): 528 529 +------------+-------------+------------+-----------------------------+ 530 | Field Size | Description | Data type | Comments | 531 +============+=============+============+=============================+ 532 | ? | count | |var_int| | Number of inventory entries | 533 +------------+-------------+------------+-----------------------------+ 534 | 32x? | inventory | inv_vect[] | Inventory vectors | 535 +------------+-------------+------------+-----------------------------+ 536 537 error 538 ^^^^^ 539 .. note:: New in version 3 540 541 This message may be silently ignored (and therefor handled like any other 542 "unknown" message). 543 544 The message is intended to inform the other node about protocol errors and 545 can be used for debugging and improving code. 546 547 .. list-table:: 548 :header-rows: 1 549 :widths: auto 550 551 * - Field Size 552 - Description 553 - Data type 554 - Comments 555 * - 1+ 556 - fatal 557 - |var_int| 558 - This qualifies the error. If set to 0, than its just a "warning". 559 You can expect, everything still worked fine. If set to 1, than 560 it's an error, so you may expect, something was going wrong 561 (e.g. an object got lost). If set to 2, it's a fatal error. The node 562 will drop the line for that error and maybe ban you for some time. 563 * - 1+ 564 - ban time 565 - var_int 566 - If the error is fatal, you can specify the ban time in seconds, here. 567 You inform the other node, that you will not accept further connections 568 for this number of seconds. For non fatal errors this field has 569 no meaning and should be zero. 570 * - 1+ 571 - inventory vector 572 - var_str 573 - If the error is related to an object, this Variable length string 574 contains the inventory vector of that object. If the error is not 575 related to an object, this string is empty. 576 * - 1+ 577 - error text 578 - var_str 579 - A human readable string in English, which describes the error. 580 581 582 object 583 ^^^^^^ 584 585 An object is a message which is shared throughout a stream. It is the only 586 message which propagates; all others are only between two nodes. Objects have a 587 type, like 'msg', or 'broadcast'. To be a valid object, the 588 :doc:`pow` must be done. The maximum allowable length of an object 589 (not to be confused with the ``objectPayload``) is |2^18| bytes. 590 591 .. |2^18| replace:: 2\ :sup:`18`\ 592 593 .. list-table:: Message structure 594 :header-rows: 1 595 :widths: auto 596 597 * - Field Size 598 - Description 599 - Data type 600 - Comments 601 * - 8 602 - nonce 603 - uint64_t 604 - Random nonce used for the :doc:`pow` 605 * - 8 606 - expiresTime 607 - uint64_t 608 - The "end of life" time of this object (be aware, in version 2 of the 609 protocol this was the generation time). Objects shall be shared with 610 peers until its end-of-life time has been reached. The node should store 611 the inventory vector of that object for some extra period of time to 612 avoid reloading it from another node with a small time delay. The time 613 may be no further than 28 days + 3 hours in the future. 614 * - 4 615 - objectType 616 - uint32_t 617 - Four values are currently defined: 0-"getpubkey", 1-"pubkey", 2-"msg", 618 3-"broadcast". All other values are reserved. Nodes should relay objects 619 even if they use an undefined object type. 620 * - 1+ 621 - version 622 - var_int 623 - The object's version. Note that msg objects won't contain a version 624 until Sun, 16 Nov 2014 22:00:00 GMT. 625 * - 1+ 626 - stream number 627 - var_int 628 - The stream number in which this object may propagate 629 * - ? 630 - objectPayload 631 - uchar[] 632 - This field varies depending on the object type; see below. 633 634 635 Unsupported messages 636 ^^^^^^^^^^^^^^^^^^^^ 637 638 If a node receives an unknown message it **must** silently ignore it. This is 639 for further extensions of the protocol with other messages. Nodes that don't 640 understand such a new message type shall be able to work correct with the 641 message types they understand. 642 643 Maybe some version 2 nodes did already implement it that way, but in version 3 644 it is **part of the protocol specification**, that a node **must** 645 silently ignore unsupported messages. 646 647 648 Object types 649 ------------ 650 651 Here are the payloads for various object types. 652 653 getpubkey 654 ^^^^^^^^^ 655 656 When a node has the hash of a public key (from an address) but not the public 657 key itself, it must send out a request for the public key. 658 659 .. list-table:: 660 :header-rows: 1 661 :widths: auto 662 663 * - Field Size 664 - Description 665 - Data type 666 - Comments 667 * - 20 668 - ripe 669 - uchar[] 670 - The ripemd hash of the public key. This field is only included when the 671 address version is <= 3. 672 * - 32 673 - tag 674 - uchar[] 675 - The tag derived from the address version, stream number, and ripe. This 676 field is only included when the address version is >= 4. 677 678 pubkey 679 ^^^^^^ 680 681 A version 2 pubkey. This is still in use and supported by current clients but 682 *new* v2 addresses are not generated by clients. 683 684 .. list-table:: 685 :header-rows: 1 686 :widths: auto 687 688 * - Field Size 689 - Description 690 - Data type 691 - Comments 692 * - 4 693 - |behavior_bitfield| 694 - uint32_t 695 - A bitfield of optional behaviors and features that can be expected from 696 the node receiving the message. 697 * - 64 698 - public signing key 699 - uchar[] 700 - The ECC public key used for signing (uncompressed format; 701 normally prepended with \x04 ) 702 * - 64 703 - public encryption key 704 - uchar[] 705 - The ECC public key used for encryption (uncompressed format; 706 normally prepended with \x04 ) 707 708 .. list-table:: A version 3 pubkey 709 :header-rows: 1 710 :widths: auto 711 712 * - Field Size 713 - Description 714 - Data type 715 - Comments 716 * - 4 717 - |behavior_bitfield| 718 - uint32_t 719 - A bitfield of optional behaviors and features that can be expected from 720 the node receiving the message. 721 * - 64 722 - public signing key 723 - uchar[] 724 - The ECC public key used for signing (uncompressed format; 725 normally prepended with \x04 ) 726 * - 64 727 - public encryption key 728 - uchar[] 729 - The ECC public key used for encryption (uncompressed format; 730 normally prepended with \x04 ) 731 * - 1+ 732 - nonce_trials_per_byte 733 - var_int 734 - Used to calculate the difficulty target of messages accepted by this 735 node. The higher this value, the more difficult the Proof of Work must 736 be before this individual will accept the message. This number is the 737 average number of nonce trials a node will have to perform to meet the 738 Proof of Work requirement. 1000 is the network minimum so any lower 739 values will be automatically raised to 1000. 740 * - 1+ 741 - extra_bytes 742 - var_int 743 - Used to calculate the difficulty target of messages accepted by this 744 node. The higher this value, the more difficult the Proof of Work must 745 be before this individual will accept the message. This number is added 746 to the data length to make sending small messages more difficult. 747 1000 is the network minimum so any lower values will be automatically 748 raised to 1000. 749 * - 1+ 750 - sig_length 751 - var_int 752 - Length of the signature 753 * - sig_length 754 - signature 755 - uchar[] 756 - The ECDSA signature which, as of protocol v3, covers the object 757 header starting with the time, appended with the data described in this 758 table down to the extra_bytes. 759 760 .. list-table:: A version 4 pubkey 761 :header-rows: 1 762 :widths: auto 763 764 * - Field Size 765 - Description 766 - Data type 767 - Comments 768 * - 32 769 - tag 770 - uchar[] 771 - The tag, made up of bytes 32-64 of the double hash of the address data 772 (see example python code below) 773 * - ? 774 - encrypted 775 - uchar[] 776 - Encrypted pubkey data. 777 778 When version 4 pubkeys are created, most of the data in the pubkey is encrypted. 779 This is done in such a way that only someone who has the Bitmessage address 780 which corresponds to a pubkey can decrypt and use that pubkey. This prevents 781 people from gathering pubkeys sent around the network and using the data from 782 them to create messages to be used in spam or in flooding attacks. 783 784 In order to encrypt the pubkey data, a double SHA-512 hash is calculated from 785 the address version number, stream number, and ripe hash of the Bitmessage 786 address that the pubkey corresponds to. The first 32 bytes of this hash are used 787 to create a public and private key pair with which to encrypt and decrypt the 788 pubkey data, using the same algorithm as message encryption 789 (see :doc:`encryption`). The remaining 32 bytes of this hash are added to the 790 unencrypted part of the pubkey and used as a tag, as above. This allows nodes to 791 determine which pubkey to decrypt when they wish to send a message. 792 793 In PyBitmessage, the double hash of the address data is calculated using the 794 python code below: 795 796 .. code-block:: python 797 798 doubleHashOfAddressData = hashlib.sha512(hashlib.sha512( 799 encodeVarint(addressVersionNumber) + encodeVarint(streamNumber) + hash 800 ).digest()).digest() 801 802 803 .. list-table:: Encrypted data in version 4 pubkeys: 804 :header-rows: 1 805 :widths: auto 806 807 * - Field Size 808 - Description 809 - Data type 810 - Comments 811 * - 4 812 - |behavior_bitfield| 813 - uint32_t 814 - A bitfield of optional behaviors and features that can be expected from 815 the node receiving the message. 816 * - 64 817 - public signing key 818 - uchar[] 819 - The ECC public key used for signing (uncompressed format; 820 normally prepended with \x04 ) 821 * - 64 822 - public encryption key 823 - uchar[] 824 - The ECC public key used for encryption (uncompressed format; 825 normally prepended with \x04 ) 826 * - 1+ 827 - nonce_trials_per_byte 828 - var_int 829 - Used to calculate the difficulty target of messages accepted by this 830 node. The higher this value, the more difficult the Proof of Work must 831 be before this individual will accept the message. This number is the 832 average number of nonce trials a node will have to perform to meet the 833 Proof of Work requirement. 1000 is the network minimum so any lower 834 values will be automatically raised to 1000. 835 * - 1+ 836 - extra_bytes 837 - var_int 838 - Used to calculate the difficulty target of messages accepted by this 839 node. The higher this value, the more difficult the Proof of Work must 840 be before this individual will accept the message. This number is added 841 to the data length to make sending small messages more difficult. 842 1000 is the network minimum so any lower values will be automatically 843 raised to 1000. 844 * - 1+ 845 - sig_length 846 - var_int 847 - Length of the signature 848 * - sig_length 849 - signature 850 - uchar[] 851 - The ECDSA signature which covers everything from the object header 852 starting with the time, then appended with the decrypted data down to 853 the extra_bytes. This was changed in protocol v3. 854 855 msg 856 ^^^ 857 858 Used for person-to-person messages. Note that msg objects won't contain a 859 version in the object header until Sun, 16 Nov 2014 22:00:00 GMT. 860 861 .. list-table:: 862 :header-rows: 1 863 :widths: auto 864 865 * - Field Size 866 - Description 867 - Data type 868 - Comments 869 * - ? 870 - encrypted 871 - uchar[] 872 - Encrypted data. See `Encrypted payload`_. 873 See also `Unencrypted Message Data`_ 874 875 broadcast 876 ^^^^^^^^^ 877 878 Users who are subscribed to the sending address will see the message appear in 879 their inbox. Broadcasts are version 4 or 5. 880 881 Pubkey objects and v5 broadcast objects are encrypted the same way: The data 882 encoded in the sender's Bitmessage address is hashed twice. The first 32 bytes 883 of the resulting hash constitutes the "private" encryption key and the last 884 32 bytes constitute a **tag** so that anyone listening can easily decide if 885 this particular message is interesting. The sender calculates the public key 886 from the private key and then encrypts the object with this public key. Thus 887 anyone who knows the Bitmessage address of the sender of a broadcast or pubkey 888 object can decrypt it. 889 890 The version of broadcast objects was previously 2 or 3 but was changed to 4 or 891 5 for protocol v3. Having a broadcast version of 5 indicates that a tag is used 892 which, in turn, is used when the sender's address version is >=4. 893 894 .. list-table:: 895 :header-rows: 1 896 :widths: auto 897 898 * - Field Size 899 - Description 900 - Data type 901 - Comments 902 * - 32 903 - tag 904 - uchar[] 905 - The tag. This field is new and only included when the broadcast version 906 is >= 5. Changed in protocol v3 907 * - ? 908 - encrypted 909 - uchar[] 910 - Encrypted broadcast data. The keys are derived as described in the 911 paragraph above. See Encrypted payload for details about the encryption 912 algorithm itself. 913 914 Unencrypted data format: 915 916 .. list-table:: 917 :header-rows: 1 918 :widths: auto 919 920 * - Field Size 921 - Description 922 - Data type 923 - Comments 924 * - 1+ 925 - broadcast version 926 - var_int 927 - The version number of this broadcast protocol message which is equal 928 to 2 or 3. This is included here so that it can be signed. This is 929 no longer included in protocol v3 930 * - 1+ 931 - address version 932 - var_int 933 - The sender's address version 934 * - 1+ 935 - stream number 936 - var_int 937 - The sender's stream number 938 * - 4 939 - |behavior_bitfield| 940 - uint32_t 941 - A bitfield of optional behaviors and features that can be expected from 942 the owner of this pubkey. 943 * - 64 944 - public signing key 945 - uchar[] 946 - The ECC public key used for signing (uncompressed format; 947 normally prepended with \x04) 948 * - 64 949 - public encryption key 950 - uchar[] 951 - The ECC public key used for encryption (uncompressed format; 952 normally prepended with \x04) 953 * - 1+ 954 - nonce_trials_per_byte 955 - var_int 956 - Used to calculate the difficulty target of messages accepted by this 957 node. The higher this value, the more difficult the Proof of Work must 958 be before this individual will accept the message. This number is the 959 average number of nonce trials a node will have to perform to meet the 960 Proof of Work requirement. 1000 is the network minimum so any lower 961 values will be automatically raised to 1000. This field is new and is 962 only included when the address_version >= 3. 963 * - 1+ 964 - extra_bytes 965 - var_int 966 - Used to calculate the difficulty target of messages accepted by this 967 node. The higher this value, the more difficult the Proof of Work must 968 be before this individual will accept the message. This number is added 969 to the data length to make sending small messages more difficult. 970 1000 is the network minimum so any lower values will be automatically 971 raised to 1000. This field is new and is only included when the 972 address_version >= 3. 973 * - 1+ 974 - encoding 975 - var_int 976 - The encoding type of the message 977 * - 1+ 978 - messageLength 979 - var_int 980 - The message length in bytes 981 * - messageLength 982 - message 983 - uchar[] 984 - The message 985 * - 1+ 986 - sig_length 987 - var_int 988 - Length of the signature 989 * - sig_length 990 - signature 991 - uchar[] 992 - The signature which did cover the unencrypted data from the broadcast 993 version down through the message. In protocol v3, it covers the 994 unencrypted object header starting with the time, all appended with 995 the decrypted data. 996 997 .. |behavior_bitfield| replace:: :ref:`behavior bitfield <behavior-bitfield>`