1  .\"
  2  .\" FreeBSD pkg - a next generation package for the installation and maintenance
  3  .\" of non-core utilities.
  4  .\"
  5  .\" Redistribution and use in source and binary forms, with or without
  6  .\" modification, are permitted provided that the following conditions
  7  .\" are met:
  8  .\" 1. Redistributions of source code must retain the above copyright
  9  .\"    notice, this list of conditions and the following disclaimer.
 10  .\" 2. Redistributions in binary form must reproduce the above copyright
 11  .\"    notice, this list of conditions and the following disclaimer in the
 12  .\"    documentation and/or other materials provided with the distribution.
 13  .\"
 14  .\"
 15  .\"     @(#)pkg.8
 16  .\"
 17  .Dd January 17, 2021
 18  .Dt PKG-REPO 8
 19  .Os
 20  .Sh NAME
 21  .Nm "pkg repo"
 22  .Nd create a package repository catalogue
 23  .Sh SYNOPSIS
 24  .Nm
 25  .Op Fl hlqs
 26  .Op Fl g Ar groups
 27  .Op Fl m Ar meta-file
 28  .Op Fl o Ar output-dir
 29  .Ao Ar repo-path Ac Op Ar signer-type: Ns Ao Ar keyfile Ac | signing_command: Ao Ar the command Ac
 30  .Pp
 31  .Nm
 32  .Op Cm --{hash,list-files,quiet,symlink}
 33  .Op Cm --groups Ar groups
 34  .Op Cm --meta-file Ar meta-file
 35  .Op Cm --output-dir Ar output-dir
 36  .Ao Ar repo-path Ac Op Ar signer-type: Ns Ao Ar keyfile Ac | signing_command: Ao Ar the command Ac
 37  .Sh DESCRIPTION
 38  .Nm
 39  is used to create a catalogue of the available
 40  packages in a repository.
 41  .Nm
 42  catalogues are necessary for sharing your package repository,
 43  and are intrinsic to the operation of
 44  .Nm "pkg install"
 45  or
 46  .Nm "pkg upgrade" .
 47  .Pp
 48  The current repository format is version 2.
 49  The repository files created by
 50  .Nm
 51  consist of a UCL metadata file and a set of compressed
 52  .Pa .pkg
 53  archives stored at the top level of the repository filesystem.
 54  .Pp
 55  .Pa meta.conf
 56  is a plain-text UCL file that must exist at the apex of the
 57  repository filesystem.
 58  It describes the repository version, compression format, and the
 59  names of the catalogue archives.
 60  See
 61  .Xr pkg-repository 5
 62  for a detailed description of its fields.
 63  .Pp
 64  .Pa packagesite.pkg
 65  is a deprecated compressed archive containing
 66  .Pa packagesite.yaml ,
 67  which lists the metadata for each of the packages in the repository.
 68  Each package manifest is represented as a single-line compact JSON
 69  text, with manifests separated by newlines.
 70  It is still generated for backward compatibility but is superseded by
 71  .Pa data.pkg .
 72  .Pp
 73  .Pa data.pkg
 74  is the primary catalogue archive.
 75  It contains a JSON file with package manifests, package group definitions, and
 76  expired package entries.
 77  See
 78  .Xr pkg-repository 5
 79  for details.
 80  .Pp
 81  .Pa filesite.pkg
 82  is an optional compressed archive containing
 83  .Pa filesite.yaml ,
 84  a database of all files present in all packages in the repository.
 85  It is only generated when the
 86  .Fl l
 87  flag is used.
 88  .Pp
 89  The compressed archives may also contain cryptographic signatures
 90  when the signing mechanism of
 91  .Nm
 92  is enabled.
 93  .Pp
 94  Repository users download these files to their local machines, where
 95  they are processed into per-repository sqlite databases for fast
 96  lookup of available packages by programs such as
 97  .Xr pkg-install 8 .
 98  .Pp
 99  To create a package repository catalogue, specify the top-level
100  directory beneath which all the packages are stored as
101  .Ar repo-path .
102  .Nm
103  will search the filesystem beneath
104  .Ar repo-path
105  to find all
106  .Pa .pkg
107  files it contains.
108  Directories starting with
109  .Sq \&.
110  or named
111  .Pa Latest
112  are not traversed.
113  .Pp
114  The repository files will be created in the top-level repository
115  directory unless relocated by specifying
116  .Fl o Ar output-dir
117  or
118  .Cm --output-dir Ar output-dir .
119  .Pp
120  Optionally, the repository catalogue may be cryptographically signed.
121  This is enabled either by specifying the path to a private key as the
122  .Ar keyfile
123  argument or by using an external command.
124  When a
125  .Ar keyfile
126  is being used, it may be prefixed by the signer type.
127  Currently, this may be one of
128  .Sy rsa ,
129  .Sy ecdsa ,
130  or
131  .Sy eddsa .
132  .Sy ecc
133  is also accepted as an alias of
134  .Sy eddsa .
135  Keys for the
136  .Sy rsa
137  and
138  .Sy ecdsa
139  signers may be generated by OpenSSL or by
140  .Xr pkg-key 8 .
141  Keys for the
142  .Dq eddsa
143  signer may only be generated by
144  .Xr pkg-key 8 .
145  .Pp
146  If the
147  .Ar key
148  is used, a hash of the repository is signed using the provided key.
149  The
150  .Sy rsa
151  signer will sign the SHA256 hash of the repository, while the
152  .Sy ecdsa
153  and
154  .Sy eddsa
155  signers will sign the BLAKE2 hash of the repository.
156  The signature is added into the repository catalogue.
157  The client side should use
158  .Sy SIGNATURE_TYPE
159  set to
160  .Dv PUBKEY
161  and
162  .Sy PUBKEY
163  set to a local path of the public key in its repository configuration file.
164  .Pp
165  An external command can be useful to create a signing server to keep the
166  private key separate from the repository.
167  The external command is passed the SHA256 of the repository
168  catalogue on its stdin.
169  It should output the following format:
170  .Bd -literal -offset indent
171  TYPE
172  signer type here (rsa, ecdsa, eddsa)
173  SIGNATURE
174  signature data here
175  CERT
176  public key data here
177  END
178  .Ed
179  .Pp
180  The
181  .Sy TYPE
182  field is optional if using
183  .Sy rsa ,
184  to remain compatible with external signing commands historically in use.
185  Note that the
186  .Sy SIGNATURE
187  field's data will may require an extra newline after it if the signature is
188  output in a binary format.
189  The
190  .Sy CERT
191  field may contain binary data, but
192  .Xr pkg 8
193  will search the tail of it for the missing
194  .Sy END
195  if it runs together.
196  .Pp
197  When using an external command, the client's
198  .Pa pkg.conf
199  must have
200  .Sy SIGNATURE_TYPE
201  set to
202  .Dv FINGERPRINTS
203  and
204  .Sy FINGERPRINTS
205  set to a directory having a
206  .Pa trusted/myrepo
207  containing a fingerprint style representation of the public key:
208  .Bd -literal -offset indent
209  function: sha256
210  fingerprint: \\"sha256_representation_of_the_public_key\\"
211  .Ed
212  .Pp
213  See the
214  .Sx EXAMPLES
215  section and
216  .Xr pkg.conf 5
217  for more information.
218  .Pp
219  Signing the catalogue is strongly recommended.
220  .Sh OPTIONS
221  The following options are supported by
222  .Nm :
223  .Bl -tag -width quiet
224  .It Fl g Ar groups , Cm --groups Ar groups
225  Set the group ownership of the repository files to
226  .Ar groups .
227  .It Fl h , Cm --hash
228  Append a short hash of the package contents to the package filename.
229  This is the same as setting the
230  .Ev PKG_REPO_HASH
231  environment variable.
232  .It Fl l , Cm --list-files
233  Generate list of all files in repo as filesite.pkg archive.
234  .It Fl m Ar meta-file , Cm --meta-file Ar meta-file
235  Use the specified file as repository meta file instead of the default settings.
236  .It Fl o Ar output-dir , Cm --output-dir Ar output-dir
237  Create the repository in the specified directory instead of the package directory.
238  .It Fl q , Cm --quiet
239  Force quiet output.
240  .It Fl s , Cm --symlink
241  Create a symlink between the hashed filename and the regular filename.
242  Only useful in combination with
243  .Fl h .
244  This is the same as setting the
245  .Ev PKG_REPO_SYMLINK
246  environment variable.
247  .El
248  .Sh FILES
249  See
250  .Xr pkg.conf 5 .
251  .Sh ENVIRONMENT
252  .Bl -tag -width "PKG_REPO_SYMLINK"
253  .It Ev PKG_REPO_HASH
254  When set, rename packages with the short hash of contents appended to the
255  filename.
256  .It Ev PKG_REPO_SYMLINK
257  When set, create a symlink between the short hash filename and the regular
258  filename.
259  .El
260  .Sh SEE ALSO
261  .Xr pkg_create 3 ,
262  .Xr pkg_printf 3 ,
263  .Xr pkg_repo_create 3 ,
264  .Xr pkg_repos 3 ,
265  .Xr pkg-keywords 5 ,
266  .Xr pkg-lua-script 5 ,
267  .Xr pkg-repository 5 ,
268  .Xr pkg-script 5 ,
269  .Xr pkg-triggers 5 ,
270  .Xr pkg.conf 5 ,
271  .Xr pkg 8 ,
272  .Xr pkg-add 8 ,
273  .Xr pkg-alias 8 ,
274  .Xr pkg-annotate 8 ,
275  .Xr pkg-audit 8 ,
276  .Xr pkg-autoremove 8 ,
277  .Xr pkg-check 8 ,
278  .Xr pkg-clean 8 ,
279  .Xr pkg-config 8 ,
280  .Xr pkg-create 8 ,
281  .Xr pkg-delete 8 ,
282  .Xr pkg-fetch 8 ,
283  .Xr pkg-help 8 ,
284  .Xr pkg-info 8 ,
285  .Xr pkg-install 8 ,
286  .Xr pkg-key 8 ,
287  .Xr pkg-lock 8 ,
288  .Xr pkg-plugins 8 ,
289  .Xr pkg-query 8 ,
290  .Xr pkg-register 8 ,
291  .Xr pkg-repositories 8 ,
292  .Xr pkg-rquery 8 ,
293  .Xr pkg-search 8 ,
294  .Xr pkg-set 8 ,
295  .Xr pkg-shell 8 ,
296  .Xr pkg-shlib 8 ,
297  .Xr pkg-ssh 8 ,
298  .Xr pkg-stats 8 ,
299  .Xr pkg-triggers 8 ,
300  .Xr pkg-unregister 8 ,
301  .Xr pkg-update 8 ,
302  .Xr pkg-updating 8 ,
303  .Xr pkg-upgrade 8 ,
304  .Xr pkg-version 8 ,
305  .Xr pkg-which 8
306  .Sh EXAMPLES
307  Create an RSA key pair:
308  .Bd -literal -offset indent
309  % openssl genrsa -out repo.key 2048
310  % chmod 0400 repo.key
311  % openssl rsa -in repo.key -out repo.pub -pubout
312  .Ed
313  .Pp
314  Create a repository and sign it with a local RSA key.
315  The public key would be shared on all client servers with
316  .Sy SIGNATURE_TYPE
317  set to
318  .Dv PUBKEY
319  and its path set via
320  .Sy PUBKEY
321  setting in the repository configuration file:
322  .Pp
323  .Dl pkg repo /usr/ports/packages repo.key
324  .Pp
325  Create a repository and sign it with an external command.
326  The client should set, via the repository configuration file,
327  .Sy SIGNATURE_TYPE
328  to
329  .Dv FINGERPRINTS
330  and
331  .Sy FINGERPRINTS
332  to a path containing a file with the SHA256 of the public key:
333  .Bd -literal -offset indent
334  # On signing server:
335  % cat > sign.sh << EOF
336  #!/bin/sh
337  read -t 2 sum
338  [ -z "$sum" ] && exit 1
339  echo SIGNATURE
340  echo -n $sum | /usr/bin/openssl dgst -sign repo.key -sha256 -binary
341  echo
342  echo CERT
343  cat repo.pub
344  echo END
345  EOF
346  
347  # On package server:
348  % pkg repo /usr/ports/packages signing_command: ssh signing-server sign.sh
349  # Generate fingerprint for sharing with clients
350  % sh -c '( echo "function: sha256"; echo "fingerprint: \\"$(sha256 -q repo.pub)\\""; ) > fingerprint'
351  # The 'fingerprint' file should be distributed to all clients.
352  
353  # On clients with FINGERPRINTS: /usr/local/etc/pkg/fingerprints/myrepo:
354  $ mkdir -p /usr/local/etc/pkg/fingerprints/myrepo/trusted
355  # Add 'fingerprint' into /usr/local/etc/pkg/fingerprints/myrepo/trusted
356  
357  .Ed
358  .Pp
359  The above examples can be repeated with OpenSSL creating a key pair for ECDSA:
360  .Bd -literal -offset indent
361  % openssl ecparam -genkey -name secp256k1 -out repo.key -outform DER
362  % chmod 0400 repo.key
363  % openssl ec -in repo.key -out repo.pub -pubout -outform DER
364  .Ed
365  .Pp
366  Prefixing the later
367  .Pa repo.key
368  reference with
369  .Dq ecdsa :
370  .Pp
371  .Dl pkg repo /usr/ports/packages ecdsa:repo.key
372  .Pp
373  The signing server example can be used mostly as-is, but with the following text
374  placed before the
375  .Sy SIGNATURE
376  section in the signing server output:
377  .Bd -literal -offset indent
378  TYPE
379  ecdsa
380  .Ed
381  .Pp
382  For EdDSA instead, create an EdDSA key pair:
383  .Bd -literal -offset indent
384  % pkg key --create -t eddsa repo.key > repo.pub
385  .Ed
386  .Pp
387  Create a repository and sign it with a local key.
388  As with the RSA example above, the public key would be shared on all client
389  servers with
390  .Sy SIGNATURE_TYPE
391  set to
392  .Dv PUBKEY
393  and its path set via the
394  .Sy PUBKEY
395  option in the repository configuration file:
396  .Pp
397  .Dl pkg repo /usr/ports/packages eddsa:repo.key
398  .Pp
399  A signing server for EdDSA could be constructed with the
400  .Fl -sign
401  option.