contributing.rst
1 ============ 2 Contributing 3 ============ 4 5 .. _hacking: 6 7 Hacking 8 ======= 9 10 All changes in your pull request **must** have 100% unit test coverage, pass 11 our `integration`_ tests, **and** be compliant with the 12 :ref:`coding style <coding-style>`. 13 14 15 Bootstrap 16 --------- 17 18 Start by :ref:`installing Let's Encrypt prerequisites 19 <prerequisites>`. Then run: 20 21 .. code-block:: shell 22 23 ./bootstrap/dev/venv.sh 24 25 Activate the virtualenv: 26 27 .. code-block:: shell 28 29 source ./$VENV_NAME/bin/activate 30 31 This step should prepend you prompt with ``($VENV_NAME)`` and save you 32 from typing ``./$VENV_NAME/bin/...``. It is also required to run some 33 of the `testing`_ tools. Virtualenv can be disabled at any time by 34 typing ``deactivate``. More information can be found in `virtualenv 35 documentation`_. 36 37 Note that packages are installed in so called *editable mode*, in 38 which any source code changes in the current working directory are 39 "live" and no further ``./bootstrap/dev/venv.sh`` or ``pip install 40 ...`` invocations are necessary while developing. 41 42 .. _`virtualenv documentation`: https://virtualenv.pypa.io 43 44 45 Testing 46 ------- 47 48 The following tools are there to help you: 49 50 - ``tox`` starts a full set of tests. Please make sure you run it 51 before submitting a new pull request. 52 53 - ``tox -e cover`` checks the test coverage only. Calling the 54 ``./tox.cover.sh`` script directly (or even ``./tox.cover.sh $pkg1 55 $pkg2 ...`` for any subpackages) might be a bit quicker, though. 56 57 - ``tox -e lint`` checks the style of the whole project, while 58 ``pylint --rcfile=.pylintrc path`` will check a single file or 59 specific directory only. 60 61 - For debugging, we recommend ``pip install ipdb`` and putting 62 ``import ipdb; ipdb.set_trace()`` statement inside the source 63 code. Alternatively, you can use Python's standard library `pdb`, 64 but you won't get TAB completion... 65 66 67 Integration 68 ~~~~~~~~~~~ 69 Mac OS X users: Run `./tests/mac-bootstrap.sh` instead of `boulder-start.sh` to 70 install dependencies, configure the environment, and start boulder. 71 72 Otherwise, install `Go`_ 1.5, libtool-ltdl, mariadb-server and 73 rabbitmq-server and then start Boulder_, an ACME CA server:: 74 75 ./tests/boulder-start.sh 76 77 The script will download, compile and run the executable; please be 78 patient - it will take some time... Once its ready, you will see 79 ``Server running, listening on 127.0.0.1:4000...``. Add an 80 ``/etc/hosts`` entry pointing ``le.wtf`` to 127.0.0.1. You may now 81 run (in a separate terminal):: 82 83 ./tests/boulder-integration.sh && echo OK || echo FAIL 84 85 If you would like to test `letsencrypt_nginx` plugin (highly 86 encouraged) make sure to install prerequisites as listed in 87 ``letsencrypt-nginx/tests/boulder-integration.sh`` and rerun 88 the integration tests suite. 89 90 .. _Boulder: https://github.com/letsencrypt/boulder 91 .. _Go: https://golang.org 92 93 94 Vagrant 95 ------- 96 97 If you are a Vagrant user, Let's Encrypt comes with a Vagrantfile that 98 automates setting up a development environment in an Ubuntu 14.04 99 LTS VM. To set it up, simply run ``vagrant up``. The repository is 100 synced to ``/vagrant``, so you can get started with: 101 102 .. code-block:: shell 103 104 vagrant ssh 105 cd /vagrant 106 ./venv/bin/pip install -r requirements.txt .[dev,docs,testing] 107 sudo ./venv/bin/letsencrypt 108 109 Support for other Linux distributions coming soon. 110 111 .. note:: 112 Unfortunately, Python distutils and, by extension, setup.py and 113 tox, use hard linking quite extensively. Hard linking is not 114 supported by the default sync filesystem in Vagrant. As a result, 115 all actions with these commands are *significantly slower* in 116 Vagrant. One potential fix is to `use NFS`_ (`related issue`_). 117 118 .. _use NFS: http://docs.vagrantup.com/v2/synced-folders/nfs.html 119 .. _related issue: https://github.com/ClusterHQ/flocker/issues/516 120 121 122 Docker 123 ------ 124 125 OSX users will probably find it easiest to set up a Docker container for 126 development. Let's Encrypt comes with a Dockerfile (``Dockerfile-dev``) 127 for doing so. To use Docker on OSX, install and setup docker-machine using the 128 instructions at https://docs.docker.com/installation/mac/. 129 130 To build the development Docker image:: 131 132 docker build -t letsencrypt -f Dockerfile-dev . 133 134 Now run tests inside the Docker image: 135 136 .. code-block:: shell 137 138 docker run -it letsencrypt bash 139 cd src 140 tox -e py27 141 142 143 Code components and layout 144 ========================== 145 146 acme 147 contains all protocol specific code 148 letsencrypt 149 all client code 150 151 152 Plugin-architecture 153 ------------------- 154 155 Let's Encrypt has a plugin architecture to facilitate support for 156 different webservers, other TLS servers, and operating systems. 157 The interfaces available for plugins to implement are defined in 158 `interfaces.py`_. 159 160 The most common kind of plugin is a "Configurator", which is likely to 161 implement the `~letsencrypt.interfaces.IAuthenticator` and 162 `~letsencrypt.interfaces.IInstaller` interfaces (though some 163 Configurators may implement just one of those). 164 165 There are also `~letsencrypt.interfaces.IDisplay` plugins, 166 which implement bindings to alternative UI libraries. 167 168 .. _interfaces.py: https://github.com/letsencrypt/letsencrypt/blob/master/letsencrypt/interfaces.py 169 170 171 Authenticators 172 -------------- 173 174 Authenticators are plugins designed to solve challenges received from 175 the ACME server. From the protocol, there are essentially two 176 different types of challenges. Challenges that must be solved by 177 individual plugins in order to satisfy domain validation (subclasses 178 of `~.DVChallenge`, i.e. `~.challenges.DVSNI`, 179 `~.challenges.SimpleHTTPS`, `~.challenges.DNS`) and continuity specific 180 challenges (subclasses of `~.ContinuityChallenge`, 181 i.e. `~.challenges.RecoveryToken`, `~.challenges.RecoveryContact`, 182 `~.challenges.ProofOfPossession`). Continuity challenges are 183 always handled by the `~.ContinuityAuthenticator`, while plugins are 184 expected to handle `~.DVChallenge` types. 185 Right now, we have two authenticator plugins, the `~.ApacheConfigurator` 186 and the `~.StandaloneAuthenticator`. The Standalone and Apache 187 authenticators only solve the `~.challenges.DVSNI` challenge currently. 188 (You can set which challenges your authenticator can handle through the 189 :meth:`~.IAuthenticator.get_chall_pref`. 190 191 (FYI: We also have a partial implementation for a `~.DNSAuthenticator` 192 in a separate branch). 193 194 195 Installer 196 --------- 197 198 Installers classes exist to actually setup the certificate and be able 199 to enhance the configuration. (Turn on HSTS, redirect to HTTPS, 200 etc). You can indicate your abilities through the 201 :meth:`~.IInstaller.supported_enhancements` call. We currently only 202 have one Installer written (still developing), `~.ApacheConfigurator`. 203 204 Installers and Authenticators will oftentimes be the same 205 class/object. Installers and Authenticators are kept separate because 206 it should be possible to use the `~.StandaloneAuthenticator` (it sets 207 up its own Python server to perform challenges) with a program that 208 cannot solve challenges itself. (Imagine MTA installers). 209 210 211 Installer Development 212 --------------------- 213 214 There are a few existing classes that may be beneficial while 215 developing a new `~letsencrypt.interfaces.IInstaller`. 216 Installers aimed to reconfigure UNIX servers may use Augeas for 217 configuration parsing and can inherit from `~.AugeasConfigurator` class 218 to handle much of the interface. Installers that are unable to use 219 Augeas may still find the `~.Reverter` class helpful in handling 220 configuration checkpoints and rollback. 221 222 223 Display 224 ~~~~~~~ 225 226 We currently offer a pythondialog and "text" mode for displays. Display 227 plugins implement the `~letsencrypt.interfaces.IDisplay` 228 interface. 229 230 231 .. _coding-style: 232 233 Coding style 234 ============ 235 236 Please: 237 238 1. **Be consistent with the rest of the code**. 239 240 2. Read `PEP 8 - Style Guide for Python Code`_. 241 242 3. Follow the `Google Python Style Guide`_, with the exception that we 243 use `Sphinx-style`_ documentation:: 244 245 def foo(arg): 246 """Short description. 247 248 :param int arg: Some number. 249 250 :returns: Argument 251 :rtype: int 252 253 """ 254 return arg 255 256 4. Remember to use ``pylint``. 257 258 .. _Google Python Style Guide: 259 https://google-styleguide.googlecode.com/svn/trunk/pyguide.html 260 .. _Sphinx-style: http://sphinx-doc.org/ 261 .. _PEP 8 - Style Guide for Python Code: 262 https://www.python.org/dev/peps/pep-0008 263 264 Submitting a pull request 265 ========================= 266 267 Steps: 268 269 1. Write your code! 270 2. Make sure your environment is set up properly and that you're in your 271 virtualenv. You can do this by running ``./bootstrap/dev/venv.sh``. 272 (this is a **very important** step) 273 3. Run ``./pep8.travis.sh`` to do a cursory check of your code style. 274 Fix any errors. 275 4. Run ``tox -e lint`` to check for pylint errors. Fix any errors. 276 5. Run ``tox`` to run the entire test suite including coverage. Fix any errors. 277 6. If your code touches communication with an ACME server/Boulder, you 278 should run the integration tests, see `integration`_. 279 7. Submit the PR. 280 281 Updating the documentation 282 ========================== 283 284 In order to generate the Sphinx documentation, run the following 285 commands: 286 287 .. code-block:: shell 288 289 make -C docs clean html 290 291 This should generate documentation in the ``docs/_build/html`` 292 directory. 293 294 .. _prerequisites: 295 296 Notes on OS dependencies 297 ======================== 298 299 OS level dependencies are managed by scripts in ``bootstrap``. Some notes 300 are provided here mainly for the :ref:`developers <hacking>` reference. 301 302 In general: 303 304 * ``sudo`` is required as a suggested way of running privileged process 305 * `Augeas`_ is required for the Python bindings 306 * ``virtualenv`` and ``pip`` are used for managing other python library 307 dependencies 308 309 .. _Augeas: http://augeas.net/ 310 .. _Virtualenv: https://virtualenv.pypa.io 311 312 Ubuntu 313 ------ 314 315 .. code-block:: shell 316 317 sudo ./bootstrap/ubuntu.sh 318 319 320 Debian 321 ------ 322 323 .. code-block:: shell 324 325 sudo ./bootstrap/debian.sh 326 327 For squeeze you will need to: 328 329 - Use ``virtualenv --no-site-packages -p python`` instead of ``-p python2``. 330 331 332 .. _`#280`: https://github.com/letsencrypt/letsencrypt/issues/280 333 334 335 Mac OSX 336 ------- 337 338 .. code-block:: shell 339 340 ./bootstrap/mac.sh 341 342 343 Fedora 344 ------ 345 346 .. code-block:: shell 347 348 sudo ./bootstrap/fedora.sh 349 350 351 Centos 7 352 -------- 353 354 .. code-block:: shell 355 356 sudo ./bootstrap/centos.sh 357 358 359 FreeBSD 360 ------- 361 362 .. code-block:: shell 363 364 sudo ./bootstrap/freebsd.sh 365 366 Bootstrap script for FreeBSD uses ``pkg`` for package installation, 367 i.e. it does not use ports. 368 369 FreeBSD by default uses ``tcsh``. In order to activate virtualenv (see 370 below), you will need a compatible shell, e.g. ``pkg install bash && 371 bash``.