1  Building Packages in HardenedBSD
  2  ================================
  3  
  4  This article will document how we at HardenedBSD have set up
  5  a local package building system for binary packages.
  6  Using Poudriere on HardenedBSD has some extra steps compared
  7  to setting it up on FreeBSD.
  8  
  9  Requirements
 10  ------------
 11  
 12  These are the requirements for our setup:
 13  
 14  1. devel/git (or devel/git-lite)
 15  1. ports-mgmt/poudriere (or ports-mgmt/poudriere-devel)
 16  
 17  The Ports Tree
 18  --------------
 19  
 20  HardenedBSD maintains its own ports repository. We've added a special
 21  hardening framework, which is easy to extend. The ports tree is on
 22  [Gitlab](https://git.hardenedbsd.org/hardenedbsd/ports). The ports
 23  tree is synced every six hours with FreeBSD's. We resolve the
 24  occasional merge conflicts usually within a twenty-four to forty-eight
 25  hour period.
 26  
 27  We install the ports tree to ```/usr/ports``` on the package building
 28  server. We tell Poudriere that we will maintain the ports tree
 29  ourselves. We've configured Poudriere to use the name ```local``` to
 30  reference our ports tree.
 31  
 32  ```
 33  # git clone https://git.hardenedbsd.org/hardenedbsd/ports.git /usr/ports
 34  # mkdir -p /usr/local/etc/poudriere.d/ports/local
 35  # echo > /usr/local/etc/poudriere.d/ports/local/method
 36  # echo /usr/ports > /usr/local/etc/poudriere.d/ports/local/mnt
 37  ```
 38  
 39  Once this is done, you should be able to run ```poudriere ports -l```
 40  and see that Poudriere knows about our ports tree:
 41  
 42  ```
 43  # poudriere ports -l
 44  PORTSTREE METHOD   TIMESTAMP PATH
 45  local     -                  /usr/ports
 46  ```
 47  
 48  If getting an error: ZPOOL variable is not set. 
 49  Make sure that your ZPOOL value is configured correctly on your poudriere.conf.
 50  
 51  
 52  Base System Source
 53  ------------------
 54  
 55  In our setup, we assume that the system has a fully populated
 56  `/usr/src`. We assume that ```make buildworld``` has been
 57  performed previously and successfully.
 58  
 59  If not, then follow these steps if on 13-STABLE:
 60  
 61  When running the 13-stable release the kernel ABI can change without extra notices.
 62  Building the world from the most recent commit will usually mean that your buildworld is ahead of your installed system. This will result in things breaking and kernel modules not loading, you don't want this.
 63  
 64  To make sure this doesn't happen we first check the commit version of the running system with
 65  ```
 66  ~ cat /var/db/hbsd-update/version 
 67  hbsd-v1200060-fb193275a276c540d1890a279e20e4515dd26aa2
 68  ```
 69  So the git commit for this version is "fb193275a276c540d1890a279e20e4515dd26aa2"
 70  
 71  
 72  Now we use this commit to build our world. Now we download the HardenedBSD source from the Github repo (it's recommended to use Github for the initial sync as they the downloads are faster) which will take some time.
 73  After the download we checkout the same version as we are runnning.
 74  
 75  sidenote always try to avoid running programs as root while connecting to the Internet when you don't need to.
 76  
 77  ```
 78  # mkdir /usr/src
 79  # chown <user>:wheel /usr/src
 80  ~ git clone --branch hardened/13-stable/master https://github.com/HardenedBSD/hardenedBSD.git /usr/src
 81  ~ cd /usr/src
 82  ~ git checkout fb193275a276c540d1890a279e20e4515dd26aa2
 83  ```
 84  
 85  for reference here is how to sync from the official HardenedBSD repo
 86  ```
 87  ~ git clone --branch hardened/13-stable/master https://git.hardenedbsd.org/hardenedbsd/HardenedBSD.git /usr/src
 88  
 89  ```
 90  
 91  Now we buildworld with the same version as the installed system.
 92  This will take several minutes, depenging on how many cores and how fast they are.
 93  
 94  ```
 95  # make -sj`sysctl -n hw.ncpu` buildworld
 96  ```
 97  
 98  
 99  
100  
101  Poudriere will use the artifacts generated from the source tree at
102  `/usr/src`, so make sure that the source tree matches your target
103  deployment for base.
104  
105  Check for existing poudriere jails:
106  
107  ```
108  # poudriere jail -l
109  ```
110  
111  
112  If it is not needed it can be destroyd with:
113  
114  ```
115  # poudriere jail -d -j stable_amd64
116  ```
117  
118  Now create the HardenedBSD STABLE/amd64 jail:
119  
120  ```
121  # poudriere jail -c -v STABLE -p local -m src=/usr/src -j stable_amd64
122  ```
123  
124  
125  Now as a test try building the pkg package with poudriere.
126  First create a file which lists ports that you want to compile and package.
127  Then start poudriere in bulk mode.
128  ```
129  # echo 'ports-mgmt/pkg' > /usr/local/etc/poudriere.d/port-list
130  # sysctl hardening.harden_rtld=0
131  # poudriere bulk -j stable_amd64 -p local -f /usr/local/etc/poudriere.d/port-list
132  ```
133  
134  If you get an error "Error: DISTFILES_CACHE directory does not exist.":
135  ```
136  mkdir -p /usr/ports/distfiles
137  ```
138  
139  If you run into issues take a look at the reference poudriere.conf file below.
140  
141  
142  Poudriere Configuration File
143  ----------------------------
144  
145  Posted below is the Poudriere config file:
146  
147  ```
148  
149  # Poudriere can optionally use ZFS for its ports/jail storage. For
150  # ZFS define ZPOOL, otherwise set NO_ZFS=yes
151  # 
152  #### ZFS
153  # The pool where poudriere will create all the filesystems it needs
154  # poudriere will use tank/${ZROOTFS} as its root
155  #
156  # You need at least 7GB of free space in this pool to have a working
157  # poudriere.
158  #
159  ZPOOL=tank
160  
161  ### NO ZFS
162  # To not use ZFS, define NO_ZFS=yes
163  #NO_ZFS=yes
164  
165  # root of the poudriere zfs filesystem, by default /poudriere
166  ZROOTFS=/poudriere/rootfs
167  
168  # the host where to download sets  for the jails setup
169  # You can specify here a host or an IP
170  # replace _PROTO_ by http or ftp
171  # replace _CHANGE_THIS_ by the hostname of the mirrors where you want to fetch
172  # by default: ftp://ftp.freebsd.org
173  #
174  # Also not that every protocols supported by fetch(1) are supported here, even
175  # file:///
176  #FREEBSD_HOST=http://0xfeedface.org/~shawn/nightlies/freebsd
177  FREEBSD_HOST=file:///src/release
178  
179  # By default the jails have no /etc/resolv.conf, you will need to set
180  # REVOLV_CONF to a file on your hosts system that will be copied has
181  # /etc/resolv.conf for the jail, except if you don't need it (using an http
182  # proxy for example)
183  RESOLV_CONF=/etc/resolv.conf
184  
185  # The directory where poudriere will store jails and ports
186  BASEFS=/poudriere/jails
187  
188  # The directory where the jail will store the packages and logs
189  # by default a zfs filesystem will be created and set to
190  # ${BASEFS}/data
191  #
192  #POUDRIERE_DATA=${BASEFS}/data
193  
194  # Use portlint to check ports sanity
195  USE_PORTLINT=no
196  
197  # When building packages, a memory device can be used to speedup the build.
198  # Only one of MFSSIZE or USE_TMPFS is supported. TMPFS is generally faster
199  # and will expand to the needed amount of RAM. MFS is a bit slower, but is
200  # more mature and can have its memory usage capped.
201  
202  # If set WRKDIRPREFIX will be mdmfs of the given size (mM or gG)
203  #MFSSIZE=32G
204  
205  # Use tmpfs(5)
206  # This can be a space-separated list of options:
207  # wrkdir    - Use tmpfs(5) for port building WRKDIRPREFIX
208  # data      - Use tmpfs(5) for poudriere cache/temp build data
209  # localbase - Use tmpfs(5) for LOCALBASE (installing ports for packaging/testing)
210  # all       - Run the entire build in memory, including builder jails.
211  # yes       - Only enables tmpfs(5) for wrkdir
212  # EXAMPLE: USE_TMPFS="wrkdir data"
213  USE_TMPFS="all"
214  
215  # If set the given directory will be used for the distfiles this allow the share
216  # the distfiles between jails and ports tree
217  DISTFILES_CACHE=/usr/ports/distfiles
218  
219  # if set the ports tree marked to use csup method will use the defined mirror
220  #CSUP_HOST=cvsup._CHANGE_THIS_.freebsd.org
221  
222  # if set the ports tree or source tree marked to use svn will use the defined
223  # mirror by default svn.FreeBSD.org
224  #SVN_HOST=svn.FreeBSD.org
225  
226  # Automatic OPTION change detection
227  # When bulk building packages, compare the options from kept packages to
228  # the current options to be built. If they differ, the existing package
229  # will be deleted and the port will be rebuilt.
230  # Valid options: yes, no, verbose
231  # verbose will display the old and new options
232  #CHECK_CHANGED_OPTIONS=verbose
233  
234  # Automatic Dependency change detection
235  # When bulk building packages, compare the dependencies from kept packages to
236  # the current dependencies for every port. If they differ, the existing package
237  # will be deleted and the port will be rebuilt. This helps catch changes such
238  # as DEFAULT_RUBY_VERSION, PERL_VERSION, WITHOUT_X11 that change dependencies
239  # for many ports.
240  # Valid options: yes, no
241  #CHECK_CHANGED_DEPS=yes
242  
243  
244  # Path to the RSA key to sign the PKGNG repo with. See pkg-repo(8)
245  #PKG_REPO_SIGNING_KEY=/usr/local/etc/ssl/keys/hardenedbsd.key
246  
247  
248  # ccache support. Supply the path to your ccache cache directory.
249  # It will be mounted into the jail and be shared among all jails.
250  #CCACHE_DIR=/var/cache/ccache
251  #
252  
253  # parallel build support.
254  #
255  # By default poudriere uses hw.ncpu to determine the number of builders.
256  # You can override this default by changing PARALLEL_JOBS here, or
257  # by specifying the -J flag to bulk/testport.
258  #
259  # Example to define PARALLEL_JOBS to one single job
260  PARALLEL_JOBS=10
261  
262  #PREPARE_PARALLEL_JOBS=5
263  
264  
265  # If set, failed builds will save the WRKDIR to ${POUDRIERE_DATA}/wrkdirs
266  # SAVE_WRKDIR=yes
267  
268  # Choose the default format for the workdir packing: could be tar,tgz,tbz,txz
269  # default is tbz
270  # WRKDIR_ARCHIVE_FORMAT=tbz
271  
272  # Disable linux support
273  NOLINUX=yes
274  
275  
276  # by default poudriere set PACKAGE_BUILDING
277  # to disable it:
278  # NO_PACKAGE_BUILDING=yes
279  
280  # If you are using a proxy define it here:
281  # export HTTP_PROXY=bla
282  # export FTP_PROXY=bla
283  #
284  # Cleanout the restricted packages
285  # NO_RESTRICTED=yes
286  
287  # By default MAKE_JOBS is disabled to allow only one process per cpu
288  # Use the following to allow it anyway
289  #ALLOW_MAKE_JOBS=yes
290  
291  
292  # Define as the URL that your POUDRIERE_DATA/logs is hosted at
293  # This will be used for giving URL hints to the HTML output when
294  # scheduling and starting builds
295  #URL_BASE=http://yourdomain.com/poudriere/
296  
297  
298  # This defines the max time (in seconds) that a command may run for a build
299  # before it is killed for taking too long. Default: 86400
300  MAX_EXECUTION_TIME=172800
301  
302  # This defines the how long (in seconds) before a command is considered to
303  # be in a runaway state for having no output on stdout. Default: 7200
304  NOHANG_TIME=57600
305  
306  URL_BASE=http:// <your custom domain here> /
307  USE_COLORS=no
308  
309  #This is the only HardenedBSD specific part when comparing to FreeBSD setups.
310  JAIL_PARAMS="hardening.pax.aslr.status=1 hardening.pax.pageexec.status=1 hardening.pax.mprotect.status=1 hardening.pax.disallow_map32bit.status=1 hardening.pax.segvguard.status=1 allow.unprivileged_proc_debug=1 hardening.harden_rtld=0"
311  
312  BUILD_AS_NON_ROOT=no
313  
314  ALLOW_MAKE_JOBS_PACKAGES="libreoffice* pkg chromium* iridium* ocaml-camomile*"
315  ```