1  # Build Codex
  2  
  3  ## Table of Contents
  4  
  5  - [Install developer tools](#prerequisites)
  6    - [Linux](#linux)
  7    - [macOS](#macos)
  8    - [Windows + MSYS2](#windows-msys2)
  9    - [Other](#other)
 10  - [Clone and prepare the Git repository](#repository)
 11  - [Build the executable](#executable)
 12  - [Run the example](#example-usage)
 13  
 14  **Optional**
 15  - [Run the tests](#tests)
 16  
 17  ## Prerequisites
 18  
 19  To build nim-codex, developer tools need to be installed and accessible in the OS.
 20  
 21  Instructions below correspond roughly to environmental setups in nim-codex's [CI workflow](https://github.com/codex-storage/nim-codex/blob/master/.github/workflows/ci.yml) and are known to work.
 22  
 23  Other approaches may be viable. On macOS, some users may prefer [MacPorts](https://www.macports.org/) to [Homebrew](https://brew.sh/). On Windows, rather than use MSYS2, some users may prefer to install developer tools with [winget](https://docs.microsoft.com/en-us/windows/package-manager/winget/), [Scoop](https://scoop.sh/), or [Chocolatey](https://chocolatey.org/), or download installers for e.g. Make and CMake while otherwise relying on official Windows developer tools. Community contributions to these docs and our build system are welcome!
 24  
 25  ### Rust
 26  
 27  The current implementation of Codex's zero-knowledge proving circuit requires the installation of rust v1.79.0 or greater. Be sure to install it for your OS and add it to your terminal's path such that the command `cargo --version` gives a compatible version.
 28  
 29  ### Linux
 30  
 31  > [!WARNING]
 32  > Linux builds currently require gcc $\leq$ 13. If this is not an option in your
 33  > system, you can try [building within Docker](#building-within-docker) as a workaround.
 34  
 35  *Package manager commands may require `sudo` depending on OS setup.*
 36  
 37  On a bare bones installation of Debian (or a distribution derived from Debian, such as Ubuntu), run
 38  
 39  ```shell
 40  apt-get update && apt-get install build-essential cmake curl git rustc cargo
 41  ```
 42  
 43  Non-Debian distributions have different package managers: `apk`, `dnf`, `pacman`, `rpm`, `yum`, etc.
 44  
 45  For example, on a bare bones installation of Fedora, run
 46  
 47  ```shell
 48  dnf install @development-tools cmake gcc-c++ rust cargo
 49  ```
 50  
 51  In case your distribution does not provide required Rust version, we may install it using [rustup](https://www.rust-lang.org/tools/install)
 52  ```shell
 53  curl --proto '=https' --tlsv1.2 -sSf https://sh.rustup.rs/ | sh -s -- --default-toolchain=1.79.0 -y
 54  
 55  . "$HOME/.cargo/env"
 56  ```
 57  
 58  Note that you will currently not be able to build Codex with gcc 14. To verify that 
 59  you have a supported version, run:
 60  
 61  ```shell
 62  gcc --version
 63  ```
 64  
 65  If you get a number that starts with 14 (e.g. `14.2.0`), then you need to either
 66  downgrade, or try a workaround like [building within Docker](#building-within-docker).
 67  
 68  ### macOS
 69  
 70  Install the [Xcode Command Line Tools](https://mac.install.guide/commandlinetools/index.html) by opening a terminal and running
 71  ```shell
 72  xcode-select --install
 73  ```
 74  
 75  Install [Homebrew (`brew`)](https://brew.sh/) and in a new terminal run
 76  ```shell
 77  brew install bash cmake rust
 78  ```
 79  
 80  Check that `PATH` is setup correctly
 81  ```shell
 82  which bash cmake
 83  
 84  # /usr/local/bin/bash
 85  # /usr/local/bin/cmake
 86  ```
 87  
 88  ### Windows + MSYS2
 89  
 90  *Instructions below assume the OS is 64-bit Windows and that the hardware or VM is [x86-64](https://en.wikipedia.org/wiki/X86-64) compatible.*
 91  
 92  Download and run the installer from [msys2.org](https://www.msys2.org/).
 93  
 94  Launch an MSYS2 [environment](https://www.msys2.org/docs/environments/). UCRT64 is generally recommended: from the Windows *Start menu* select `MSYS2 MinGW UCRT x64`.
 95  
 96  Assuming a UCRT64 environment, in Bash run
 97  ```shell
 98  pacman -Suy
 99  pacman -S base-devel git unzip mingw-w64-ucrt-x86_64-toolchain mingw-w64-ucrt-x86_64-cmake mingw-w64-ucrt-x86_64-rust
100  ```
101  
102  We should downgrade GCC to version 13 [^gcc-14]
103  ```shell
104  pacman -U --noconfirm \
105    https://repo.msys2.org/mingw/ucrt64/mingw-w64-ucrt-x86_64-gcc-13.2.0-6-any.pkg.tar.zst \
106    https://repo.msys2.org/mingw/ucrt64/mingw-w64-ucrt-x86_64-gcc-libs-13.2.0-6-any.pkg.tar.zst
107  ```
108  
109  #### Optional: VSCode Terminal integration
110  
111  You can link the MSYS2-UCRT64 terminal into VSCode by modifying the configuration file as shown below.
112  File: `C:/Users/<username>/AppData/Roaming/Code/User/settings.json`
113  ```json
114  {
115      ...
116      "terminal.integrated.profiles.windows": {
117        ...
118        "MSYS2-UCRT64": {
119          "path": "C:\\msys64\\usr\\bin\\bash.exe",
120          "args": [
121            "--login",
122            "-i"
123          ],
124          "env": {
125            "MSYSTEM": "UCRT64",
126            "CHERE_INVOKING": "1",
127            "MSYS2_PATH_TYPE": "inherit"
128          }
129        }
130      }
131  }
132  ```
133  
134  ### Other
135  
136  It is possible that nim-codex can be built and run on other platforms supported by the [Nim](https://nim-lang.org/) language: BSD family, older versions of Windows, etc. There has not been sufficient experimentation with nim-codex on such platforms, so instructions are not provided. Community contributions to these docs and our build system are welcome!
137  
138  ## Repository
139  
140  In Bash run
141  ```shell
142  git clone https://github.com/codex-storage/nim-codex.git repos/nim-codex && cd repos/nim-codex
143  ```
144  
145  nim-codex uses the [nimbus-build-system](https://github.com/status-im/nimbus-build-system), so next run
146  ```shell
147  make update
148  ```
149  
150  This step can take a while to complete because by default it builds the [Nim compiler](https://nim-lang.org/docs/nimc.html).
151  
152  To see more output from `make` pass `V=1`. This works for all `make` targets in projects using the nimbus-build-system
153  ```shell
154  make V=1 update
155  ```
156  
157  ## Executable
158  
159  In Bash run
160  ```shell
161  make
162  ```
163  
164  The default `make` target creates the `build/codex` executable.
165  
166  ## Tools
167  
168  ### Circuit download tool
169  
170  To build the circuit download tool located in `tools/cirdl` run:
171  
172  ```shell
173  make cirdl
174  ```
175  
176  ## Example usage
177  
178  See the instructions in the [Quick Start](/learn/quick-start).
179  
180  ## Tests
181  
182  In Bash run
183  ```shell
184  make test
185  ```
186  
187  ### testAll
188  
189  #### Prerequisites
190  
191  To run the integration tests, an Ethereum test node is required. Follow these instructions to set it up.
192  
193  ##### Windows (do this before 'All platforms')
194  
195  1. Download and install Visual Studio 2017 or newer. (Not VSCode!) In the Workloads overview, enable `Desktop development with C++`. ( https://visualstudio.microsoft.com )
196  
197  ##### All platforms
198  
199  1. Install NodeJS (tested with v18.14.0), consider using NVM as a version manager. [Node Version Manager (`nvm`)](https://github.com/nvm-sh/nvm#readme)
200  1. Open a terminal
201  1. Go to the vendor/codex-contracts-eth folder: `cd /<git-root>/vendor/codex-contracts-eth/`
202  1. `npm install` -> Should complete with the number of packages added and an overview of known vulnerabilities.
203  1. `npm test` -> Should output test results. May take a minute.
204  
205  Before the integration tests are started, you must start the Ethereum test node manually.
206  1. Open a terminal
207  1. Go to the vendor/codex-contracts-eth folder: `cd /<git-root>/vendor/codex-contracts-eth/`
208  1. `npm start` -> This should launch Hardhat, and output a number of keys and a warning message.
209  
210  #### Run
211  
212  The `testAll` target runs the same tests as `make test` and also runs tests for nim-codex's Ethereum contracts, as well a basic suite of integration tests.
213  
214  To run `make testAll`.
215  
216  Use a new terminal to run:
217  ```shell
218  make testAll
219  ```
220  
221  ## Building Within Docker
222  
223  For the specific case of Linux distributions which ship with gcc 14
224  and a downgrade to 13 is not possible/desirable, building within a Docker
225  container and pulling the binaries out by copying or mounting remains an
226  option; e.g.:
227  
228  ```bash=
229  # Clone original repo.
230  git clone https://github.com/codex-storage/nim-codex
231  
232  # Build inside docker
233  docker build -t codexstorage/nim-codex:latest -f nim-codex/docker/codex.Dockerfile nim-codex
234  
235  # Extract executable
236  docker create --name=codex-build codexstorage/nim-codex:latest
237  docker cp codex-build:/usr/local/bin/codex ./codex
238  docker cp codex-build:/usr/local/bin/cirdl ./cirdl
239  ```
240  
241  and voilà, you should have the binaries available in the current folder.