/ README
README
1 Arching Kaos Tools 2 ================== 3 4 > Warning: The tools and the whole project is in an experimental state. 5 > Everything is subject to change. 6 7 > Hint: Find the "Script usage" section for sum up of the scripts 8 9 > Disclaimer: Any use of "AI" tools upon this repository, is STRICTLY forbidden. 10 > This affects all commits and branches. 11 12 Introduction 13 ------------ 14 Tools that help to create a blockchain called `zchain` by preparing small JSON 15 objects that are pointing one to another in the following way: 16 ``` 17 ZBLOCK = BLOCK + SIGNATURE 18 BLOCK = ACTION + DATA + DATA_SIGNATURE + TIMESTAMP + GPG + PREVIOUS_ZBLOCK 19 DATA = IPFS_CID + SIGNATURE ( + key-pair values that are based on ACTION ) 20 ``` 21 Most of the times, `IPFS_CID` in `DATA` is expected to be a file. The module 22 that is embedded inside the `ACTION` field is responsible for handling it 23 properly. 24 25 For the zchain to be valid, all the signatures must be verifiable against the 26 provided `GPG` key that is included in the `BLOCK`. 27 28 For more information, refer to the `arching-kaos-infochain` repository. 29 30 Previous developments 31 --------------------- 32 1. A proof of work blockchain is introduced to synchronize the zchains among 33 them. 34 2. A miner prototype is introduced as well. 35 3. From a directory full of blocks, find the latest block of the longest chain 36 4. API for calling some of the bash scripts 37 38 Description 39 ----------- 40 This is an installable repository which provides various tools for running 41 Arching Kaos and using it. 42 43 Generic tools: 44 - `ak` - The tools wrapper 45 - `ak zchain --crawl` - Can be used to crawl zchains. 46 - `ak zchain --rebase` - Can be used to rebase your zchain to a previous zblock. 47 - `ak zchain --reset` - Can be used to reset your zchain to zgenesis. 48 49 Modules: 50 You can use previously known modules or make new ones and put them into your 51 `modules/` directory. The tree has to be the bare minimum of: 52 ``` 53 modules/ 54 └─ <module name> 55 ├─ lib.sh 56 └─ main.sh 57 ``` 58 - `mixtapes` and `news` - Can be used to add content to your zchain. 59 - `profile` - Can be used to build up a profile of the contributor. 60 61 Based on your GPG, zchain and other values provided by `profile`, you can create 62 with `akconfig` a pointer to some basic information that you can share with 63 others in order for them to crawl your zchain or view your "profile". 64 65 There are more tools available under the `bin` folder. Make sure you deeply 66 understand what they are doing before using them. 67 68 Changes are mentioned in `git log` of the repo, as well as in CHANGELOG file. 69 70 Requirements 71 ------------ 72 Some Linux machine will propably work. It's tested under Fedora Linux. External 73 programs used are: 74 75 - bash (v5.1.8) 76 - gpg (v2.3.4) 77 - curl (v7.79.1) 78 - git (v2.34.1) 79 - which (v2.21) 80 - rlwrap (v0.46.2) 81 - jq (v1.6) 82 - nodejs (v16+) 83 - npm (v9.5.0) 84 - npx (v9.5.0) 85 86 How to install the repository 87 ----------------------------- 88 Write on your terminal: 89 ``` 90 git clone https://github.com/arching-kaos/arching-kaos-tools 91 cd arching-kaos-tools 92 ./install.sh 93 ``` 94 95 If you encounter any errors, please open an issue. 96 97 For more extended description on the installation process, refer to `INSTALL` 98 file. 99 100 Update 101 ------ 102 Navigate to your cloned repository and execute the following commands: 103 104 ``` 105 git pull 106 ./update.sh 107 ``` 108 109 Uninstall 110 --------- 111 > WARNING: It will REMOVE the whole $HOME/.arching-kaos directory but your GPG 112 > keyring. Proceed with caution! Back up if you want to. 113 114 Navigate to your cloned repository and execute the following command: 115 116 ``` 117 ./uninstall.sh 118 ``` 119 This will output an archive with your aknet-gpg-keyring to your $HOME folder. 120 121 Podman (or Docker) 122 ------------------ 123 There is a ContainerFile that you can use to build an image which you can then 124 deploy in a container. 125 126 Use: 127 128 ``` 129 podman build -f ContainerFile -t arching-kaos-tools . 130 ``` 131 132 Specialized ContainerFiles are available in `./podman/` for various 133 distributions: 134 - alpine 135 - archlinux 136 - debian 137 - fedora 138 - opensuse 139 140 More over there are two scripts for testing with podman: 141 1. `./test_with_podman.sh` - build one or all images 142 2. `./full_test_with_podman.sh` - build one image and bash in it 143 144 TODO 145 ---- 146 Next things to come are: 147 - [ ] - zblock manipulator to fix wrong previous block references and repack 148 zblocks. 149 - [X] - zchain rebase-like procedure to move the zblocks to another seed or set 150 another previous block (e.g. join chains) 151 - [x] - Clean up installation and filesystem usage 152 - [x] - Log rotate to gzip archives 153 - [ ] - Filters for log searching for IPFS hashes or names 154 - [ ] - ~~Use of IPFS file system to store/pin sub chains, previous chains or~~ 155 ~~other chains~~ 156 157 Concepts under thought 158 ---------------------- 159 - Base chain, derived from personal ones. 160 - Daemon to monitor chains' states, update and sign new zblocks. 161 - Find a minimum agreement of what is valid or not. (important) 162 - Spam defence (e.g. how to protect the network from abuse cases) (important) 163 - Reduce size of all parts of chain (stringify JSON) (improvement) 164 165 Scripts' description 166 -------------------- 167 168 Utilities 169 --------- 170 - ak-log # Logging tools 171 - ak-clean # Cleans up temporary folders created by these tools 172 - ak-config # Publish your configuration to IPNS 173 - ak-get-zlatest # Returns zlatest ZBLOCK according to IPFS FS 174 - ak-gpg # Return your GPG key as IPFS CID 175 - ak-network # Network Tools (connect to, scan for, dump peers) 176 - ak-node-info # Returns IPFS CID or IPNS key for your online config 177 - ak-pkg # Package manager 178 - ak-fs # File system 179 - ak-ns # Name system 180 - ak-sblock # sblock tools 181 - ak-schain # schain tools 182 - ak-zblock # zblock tools 183 - ak-zchain # zchain tools 184 185 Modules 186 ------- 187 - comments # References a comment to a ZBLOCK 188 - files # Adds a file to the ZCHAIN 189 - mixtapes # Adds a mixtape to the ZCHAIN 190 - news # Adds a news article to the ZCHAIN 191 - profile # Adds key-value pairs to the ZCHAIN 192 - reference # Adds references to the ZCHAIN 193 - repositories # Adds repositories to the ZCHAIN 194 - transactions # Prototype of transactions in the ZCHAIN 195 - smfiles # Adds a split file's map to the ZCHAIN 196 - articles PROTO Adds an article to your ZCHAIN 197 - categories PROTO Adds or references categories to the ZCHAIN 198 - folders PROTO Adds a folder to the ZCHAIN 199 - roadmap PROTO Adds a roadmap to the ZCHAIN 200 - todos PROTO Adds todo list to the ZCHAIN 201 - follow PROTO Follow IPNS keys of ZCHAIN for changes 202 - encrypted # Encrypts data and publishes to ZCHAIN 203 - markdown # Adds markdown documents to ZCHAIN 204 205 PROTO refers to a prototype implementation of the module 206 Read more at the `./MODULES` file 207 208 Experimental 209 ------------ 210 - ak-mempool # TODO 211 - ak-mine PROTO Prototype miner 212 - ak-get-balances # Returns a balances from SCHAIN 213 214 Libraries 215 --------- 216 - lib/_ak_checks 217 - lib/_ak_cjdns 218 - lib/_ak_coin 219 - lib/_ak_sblock 220 - lib/_ak_schain 221 - lib/_ak_smfiles 222 - lib/_ak_wallet 223 - lib/_ak_network_stellar 224 - lib/_ak_network_ipfs 225 - lib/_ak_sh 226 - lib/_ak_html 227 - lib/_ak_network 228 - lib/_ak_network_cjdns 229 - lib/_ak_config 230 - lib/_ak_node 231 - lib/_ak_irc 232 - lib/_ak_maintainance 233 - lib/_ak_script 234 - lib/_ak_fm 235 - lib/_ak_yggdrasil 236 - lib/_ak_hash_exchange 237 - lib/_ak_zchain 238 - lib/_ak_network_yggdrasil 239 - lib/_ak_utils 240 - lib/_ak_ipfs 241 - lib/_ak_settings 242 - lib/_ak_gpg 243 - lib/_ak_fs 244 - lib/_ak_ns 245 - lib/_ak_pkg 246 - lib/_ak_zblock 247 - lib/_ak_log 248 249 Wrappers 250 -------- 251 - `ak-ipfs -r` 252 - `ak-gpg -r` 253 254 Deprecated 255 ---------- 256 - ak-json2bash 257 258 259 Debugging 260 --------- 261 Setting `AK_DEBUG` shell variable to `"yes"` will make all log messages to 262 appear on your stderr (file descriptor 2). After setting this variable, run the 263 tools you want. 264 265 Example: 266 ``` console 267 AK_DEBUG="yes"; ak zchain --crawl -l 1 268 ``` 269 270 Examples 271 -------- 272 ### Add a news article ( `ak -m news --add` ) 273 274 You could use ZCHAIN with NEWS model. Or MIXTAPE model, or make your own. 275 ``` bash 276 $ ak -m news --create 277 ``` 278 279 This would pop up a vim editor for you to write a news article or whatever is 280 text or markdown format with a title. 281 282 Saving the file, will save it locally, add it to IPFS, sign it, pack detached 283 signature with metadata on a JSON object. Then a block will be created packing 284 your GPG public key, the news/add action with the JSON object and a detached 285 signature of this, timestamp and finally an entry for the previous *zblock*. 286 After that, we finally write this as a JSON object, add it to IPFS, sign it 287 and pack a *ZBLOCK*. The ZBLOCK is then published over our IPNS zchain key. 288 289 Other options... let's try help! 290 291 ``` console 292 $ ak -m news -h 293 ak-news - Module to read, create and add zblocks 294 ================================================ 295 -h, --help Prints this help message 296 -l, --local-index Prints an indexed table of your news files 297 -i, --import <file> #TODO 298 -a, --add <file> Creates a data file from the news file you point to 299 -r, --read <zblock> Reads a zblock as a news data 300 -c, --create Vim is going to pop up, you will write and save your 301 newsletter and it's going to be saved 302 -s, --specs Print specs of data block 303 ``` 304 305 Clearly there is a TODO item. Import is not working so avoid it, or fix it. 306 Add is nice, you can add an already existing file directly. `news` is the second 307 module after `mixtape`. Both modules need refactoring but they work at a level 308 that someone can be productive with these tools. 309 310 ### Explore chains ( `ak-zchain --crawl` ) 311 312 You can view your zchain as a JSON object using `enter`. There are some flags 313 in order to either view other zchains or change the depth of view ( includes or 314 ignores data object and action ). 315 316 ``` console 317 $ ak zchain --crawl -h 318 ak zchain --crawl [-N | --no-verify] [-l | --limit <number>] [zblock] 319 ak zchain --crawl [-N | --no-verify] [-l | --limit <number>] -n <zchain> 320 Usage: 321 322 --help, -h Print this help and exit 323 324 --chain <ipns-link>, -n <ipns-link> Crawl specified chain 325 326 --no-verify, -N Don't verify signatures 327 328 <ipfs-link> Specify IPFS CID for entrance 329 330 Note that combined flags don't work for now 331 Running with no flags crawls your chain based on AK_ZLATEST environment 332 variable 333 ``` 334 335 ### Connect to networks ( `ak network` ) 336 ```console 337 $ ak network 338 # ak-network - AK Nettool 339 # ======================= 340 # 341 # Network tools 342 # 343 # Usage: 344 # 345 # -h, --help Prints this help message 346 # -c, --connect [ipfs|cjdns] Connect to network(s) 347 # -p, --peers [ipfs|cjdns|stellar|yggdrasil|incoming] Outputs peers found 348 # -s, --scan [ipfs|cjdns|stellar|yggdrasil|incoming] Scan network(s) for peers 349 # 350 # Bonus: CJDNS network scanning modes 351 # 352 # -s, --scan cjdns [-w|--whole] Scan using HIA's knowledge 353 # -s, --scan cjdns [-d|--dump] Scan using CJDNS' dumpLinks (default) 354 # 355 ``` 356 357 FS tools 358 -------- 359 - ak-fs 360 361 ```bash 362 $ ak fs --help 363 # ak-fs - AKFS tools 364 # ================== 365 # 366 # Tools to add, get and cat files to/from the Arching Kaos File System 367 # 368 # Usage: 369 # 370 # -h, --help Prints this help message 371 # --add, --import <file> Adds/Imports a file to the AKFS system 372 # --get, --export <roothash> <output file> Exports a file from the AKFS system 373 # --cat <roothash> Concatenates from given hash 374 # --list Lists names and roots available 375 # --net-cat-from-map <maphash> Concatenates from map via the network 376 # --net-get-from-map <maphash> Downloads from map via the network 377 # --cfm <maphash> Concatenates from map 378 # --gfm <maphash> Get file from map 379 # 380 ``` 381 382 Package manager 383 --------------- 384 - ak-pkg 385 386 ```bash 387 $ ak pkg --help 388 # ak-pkg - Package Manager 389 # ======================== 390 # 391 # Manager of packages that can be installed and run as modules invoking the 392 # `ak -m <module>` sequence. 393 # 394 # Usage: 395 # 396 # -h, --help Prints this help message 397 # --make-releases-from-local Makes releases from all local installation's module 398 # --make-release-from-local Makes release from a local installation's module 399 # --install-from-zblock <zblock> Installs from a zblock 400 # --install-from-akfs <maphash> Installs from an AKFS map 401 # --uninstall Uninstalls a module selected from a menu 402 # 403 ``` 404 405 NS tools 406 -------- 407 - ak-ns 408 409 ```bash 410 $ ak ns --help 411 # ak-ns - Name system 412 # =================== 413 # 414 # AKNS is a name system for Arching Kaos 415 # 416 # Usage: 417 # 418 # -h, --help Prints this help message 419 # -c, --create <name> Creates a new key-pair with name 420 # -l, --list List names 421 # -ll, --list-long List names with keys 422 # -rn, --resolve-name <name> Resolves value from name 423 # -rk, --resolve-key <key> Resolves value from key 424 # -p, --publish <key> <value> Publishes value to key 425 # -pn, --publish2name <name> <value> Publishes value to name 426 # -pz, --publish-zchain Publishes zchain 427 # -pc, --publish-config Publishes config 428 # -ek, --encode-key <key> Encodes a key to Base64 429 # -dk, --decode-key <base64 key> Decodes a key from Base64 430 # 431 ns command finished 432 ``` 433 434 Donations 435 --------- 436 While this project is developed with enthusiasm and it uses the spare time of 437 the only developer that is coding for it, donations would help allocating more 438 time on the project instead of the developer looking for different occupations 439 to support their life and project. 440 441 If you do appreciate the project's goals and the developers efforts towards it, 442 take your time and consider donating some satoshis to the developer via BTC at 443 the following address 444 445 ``` 446 bc1q70rgp65t7acfgpwp74m7vdz0g4eduxm6a43gd8 447 ``` 448 449 Thank you!