Using Torrent Files to Download Content from the Codex Network.md
1 For the convenience, you can also use a torrent file when downloading the content from the Codex network. After the content has been uploaded to the Codex network, the same torrent file can be used to refer to the content on the BitTorrent network as well as on the Codex network. In the end, both will result in the same `info` attribute and the same `info` hash. 2 3 > [!note] 4 > `.torrent` files are b-encoded, thus machine-readable. For even more convenience, we also support human-readable torrent files in JSON format. 5 6 Let's create an example torrent file. To keep it compact for the purpose of the clarity of this document, lets first upload a small file to our Codex network: 7 8 ```bash 9 curl -X POST \ 10 http://localhost:8001/api/codex/v1/torrent \ 11 -H 'Content-Type: application/octet-stream' \ 12 -H 'Content-Disposition: filename="data1M.bin"' \ 13 -w '\n' \ 14 -T data1M.bin 15 1114F335440998515770ADF47E8A4626889E59D91DDE 16 ``` 17 18 Now let's retrieve the corresponding torrent manifest: 19 20 ```bash 21 export INFO_HASH=1114F335440998515770ADF47E8A4626889E59D91DDE 22 curl "http://localhost:8002/api/codex/v1/torrent/${INFO_HASH}/network/manifest" | jq 23 { 24 "infoHash": "1114F335440998515770ADF47E8A4626889E59D91DDE", 25 "torrentManifest": { 26 "info": { 27 "length": 1048576, 28 "pieceLength": 262144, 29 "pieces": [ 30 "111421FEBA308CD51E9ACF88417193A9EA60F0F84646", 31 "11143D4A8279853DA2DA355A574740217D446506E8EB", 32 "11141AD686B48B9560B15B8843FD00E7EC1B59624B09", 33 "11145015E7DA0C40350624C6B5A1FED1DB39720B726C" 34 ], 35 "name": "data1M.bin" 36 }, 37 "codexManifestCid": "zDvZRwzmAzLRUfd3NGQ5JSP42ATgeVBeY9RE1CgYTWZDtPTDZcqz" 38 } 39 } 40 ``` 41 42 The `info` part of the returned torrent manifest is already a valid torrent file in JSON encoding: 43 44 ```json 45 { 46 "info": { 47 "length": 1048576, 48 "name": "data1M.bin", 49 "pieceLength": 262144, 50 "pieces": [ 51 "111421FEBA308CD51E9ACF88417193A9EA60F0F84646", 52 "11143D4A8279853DA2DA355A574740217D446506E8EB", 53 "11141AD686B48B9560B15B8843FD00E7EC1B59624B09", 54 "11145015E7DA0C40350624C6B5A1FED1DB39720B726C" 55 ], 56 }, 57 } 58 ``` 59 60 In the JSON encoding the order of the attributes does not matter: our client will sort the attributes accordingly in order to compute the correct corresponding `info` hash. 61 62 Let's put the JSON representation of the torrent file into `data1M.bin.torrent.json`. 63 64 Because only `info` dictionary is used to compute the corresponding `info` hash, we do not have to include any other torrent attributes (e.g. `announce` attribute used by trackers). 65 66 > [!note] 67 > We currently do not support directories, thus we do not need to support the `files` attribute. 68 69 Also notice that the `name` attribute has purely advisory character - it is a suggested name to save a file or dictionary, but since we are returning the content via API, we do not have to observe this attribute. Yet, this attribute must be present and its value must be kept intact as changing it will affect the corresponding `info` hash value. 70 71 Our API allows us to use the JSON representation of the torrent file in order to retrieve the corresponding content: 72 73 ```bash 74 curl -X POST \ 75 http://localhost:8001/api/codex/v1/torrent/torrent-file \ 76 -H 'Content-Type: application/json' \ 77 -w '\n' \ 78 -T data1M.bin.torrent.json -o ${INFO_HASH}-JSON.bin 79 ``` 80 81 We can check that the fetched file content is identical to `data1M.bin`: 82 83 ```bash 84 openssl sha1 data1M.bin 85 SHA1(data1M.bin)= f0168f6ff31fd7511bdbcf7f70fecce959f321e7 86 openssl sha1 1114F335440998515770ADF47E8A4626889E59D91DDE-JSON.bin 87 SHA1(1114F335440998515770ADF47E8A4626889E59D91DDE-JSON.bin)= f0168f6ff31fd7511bdbcf7f70fecce959f321e7 88 ``` 89 90 Now let's create the corresponding binary version of the torrent file and use it to fetch the same content. 91 92 Having the torrent manifest, what we need to create the corresponding BitTorrent v1 torrent file is the content of the info dictionary. We need to change the name of the `pieceLength` attribute to `piece length` as required by the BitTorrent specs. 93 94 ```json 95 { 96 "info": { 97 "length": 1048576, 98 "name": "data1M.bin", 99 "piece length": 262144, 100 "pieces": [ 101 "111421FEBA308CD51E9ACF88417193A9EA60F0F84646", 102 "11143D4A8279853DA2DA355A574740217D446506E8EB", 103 "11141AD686B48B9560B15B8843FD00E7EC1B59624B09", 104 "11145015E7DA0C40350624C6B5A1FED1DB39720B726C" 105 ], 106 }, 107 } 108 ``` 109 110 111 112 In order to convert this human readable, JSON representation of the `info` dictionary, we can use a Python script which we show in [[B-encoding using Python]]. Our encoder will enforce the correct order of the attributes while encoding. 113 114 The resulting b-encoded Torrent file content will be: 115 116 ```bash 117 d4:infod6:lengthi1048576e4:name10:data1M.bin12:piece lengthi262144e6:pieces80:!��0�� �ψAq���`��FF=J�y�=��5ZWG@!}De��ֆ���`�[�C� ��YbK P�� @5$Ƶ����9r rlee 118 ``` 119 120 The encoder will also return the corresponding `info` hash value: 121 122 ```bash 123 f335440998515770adf47e8a4626889e59d91dde 124 ``` 125 126 It should match the `info` value we got after uploading the content. 127 128 Putting the b-encoded torrent file in `data1M.bin.torrent`, we can use it directly the retrieve the corresponding content: 129 130 ```bash 131 curl -X POST \ 132 http://localhost:8001/api/codex/v1/torrent/torrent-file \ 133 -H 'Content-Type: application/octet-stream' \ 134 -w '\n' \ 135 -T data1M.bin.torrent -o ${INFO_HASH}.bin 136 ``` 137 138 Also here we can see that the retrieved content matches the original: 139 140 ```bash 141 openssl sha1 data1M.bin 142 SHA1(data1M.bin)= f0168f6ff31fd7511bdbcf7f70fecce959f321e7 143 openssl sha1 1114F335440998515770ADF47E8A4626889E59D91DDE.bin 144 SHA1(1114F335440998515770ADF47E8A4626889E59D91DDE.bin)= f0168f6ff31fd7511bdbcf7f70fecce959f321e7 145 ```