1  # Notes on using emacs as a Python 'IDE': eply or lsp-mode
  2  
  3  There are two Python IDE environments I have used, and which this configuration
  4  can support: *lsp-mode* with *python-lsp-server* or *elpy*.
  5  
  6  Both provide advanced Python specific IDE abilities and greatly enhance the
  7  Python programming experience inside *emacs*.
  8  
  9  Both have some of the same features and this overlap means they are not fully
 10  compatible. You will have to pick between them.
 11  
 12  In general, *elpy* is a bit leaner. *lsp-mode* is slower and provides a more
 13  consistent interface between the different programming languages which
 14  *lsp-mode* supports.
 15  
 16  *lsp-mode* is a compelling choice if you use the lsp-server to enforce
 17  consistent style and code requirements across a shop/team. You simply pick a
 18  server and configure it with the standard requirements and everyone can use
 19  their own preferred editor/IDE pointed at the configured lsp server. Then
 20  everyone is writing code to the same standards.
 21  
 22  The default setup for Python uses *python-lsp-server*.
 23  
 24  ## Using *python-lsp-server*
 25  
 26  *lsp-mode* should start or connect to an existing *python-lsp-server* whenever
 27  you open any python file.
 28  
 29  If lsp-mode is running poorly, you can triage the situation with *lsp-doctor*.
 30  
 31  A good guide for using *lsp-doctor* and increasing *lsp-mode* performance is
 32  found here: <https://emacs-lsp.github.io/lsp-mode/page/performance/>.
 33  
 34  A good guide for configuring and setting up *python-lsp-server* is here:
 35  <https://emacs-lsp.github.io/lsp-mode/page/lsp-pylsp/#server>
 36  
 37  
 38  ## Enabling and using elpy
 39  
 40  See: <https://elpy.readthedocs.io/en/latest/introduction.html> for more info on *elpy*.
 41  
 42  By default, *elpy* is disabled in favor of LSP, but you can edit
 43  *custom/setup-python.el* to enable it. First, uncomment sections referencing
 44  *elpy*. and then comment all sections in all files referencing *lsp-mode* and
 45  python in *custom/setup-editing.el*.
 46  
 47  To make sure elpy is running correctly, *M-X elpy-config*. Note any *warnings*
 48  or missing packages and install them.
 49  
 50  If you are using a dedicated virtual environment per project, elpy may warn
 51  about *~/.local/bin* not being in the PATH. You will have to either: modify your PATH
 52  variable before starting emacs, or stop running emacs from the virtual
 53  environment.
 54  
 55  To modify your path, add a line like the following to your *~/.bashrc*:
 56  
 57  ```bash
 58  export PATH="$PATH:$HOME/.local/bin"
 59  ```
 60  
 61  ## Adding and configuring additional Python packages: autopep8, yapf, jedi, rope, black, flake8
 62  
 63  These pip packages provide features used by *elpy* and/or *python-lsp-server*,
 64  but are handy on their own.
 65  
 66  * *autopep8* helps elpy enforce compliance with the Python Pep-8 coding standards
 67  by underlining in red all non-compliant code. See:
 68  <http://paetzke.me/project/py-autopep8.el>.
 69  
 70  * *flake8* is the linter static analysis engine used by autopep8.
 71  
 72  * *yapf* is used as a more feature completed *autopep8* with a slightly different
 73  and more hands-on approach to enforcing code-style beyond pep8 compliance.
 74  
 75  In general, you should configure your editor to use 1 of these at a time.
 76  
 77  However, you can still install all of them on your machine and have different
 78  linting tools run them as part of your gating process-- both inside or outside
 79  of *emacs*.
 80  
 81  
 82  To install all python packages supported by Elpy:
 83  
 84  ```bash
 85  pip install autopep8 jedi rope yapf black flake8 pydocstyle
 86  ```
 87  
 88  To install all optional packages supported by *python-lsp-server*:
 89  
 90  ```bash
 91  pip install python-lsp-server[all]
 92  
 93  ```
 94  
 95  Sometimes, you may want to override some of autopep8's style settings, such as
 96  the default line length being set to only allow 80 characters, which is archaic.
 97  
 98  Another issue is that pycodestyle enforces W503 and W504 out of the box
 99  when all warnings are enabled.  These two rules actually conflict with each
100  other. The current best practice in PEP8 is to use W504, as W503 is deprecated.
101  
102  To change settings, edit *$HOME/.config/pycodestyle* and add the appropriate
103  ini-file style configuration settings.  The following changes the default line
104  length from 80 to 160 characters and turns off W503:
105  
106  ```
107  [pycodestyle]
108  max_line_length = 160
109  ignore = W503
110  ```
111  
112  To override the default pylint settings, create an rc file in your home
113  directory or project root called *.pylintrc*:
114  
115  ```
116  [MASTER]
117  
118  persistent=yes
119  
120  [FORMAT]
121  max-line-length=160
122  
123  ```
124  
125  To override settings in flake8, look for file *~/.config/flake8*:
126  ```
127  [flake8]
128  max-line-length = 160
129  ```
130  
131  ## Using the python invoke package
132  
133  Most of my Python projects use a package called *invoke* to execute the creation
134  of build and release targets. *invoke* is a "task execution tool and library."
135  It is similar in purpose to *make*, *maven* or Ruby *rake*.
136  
137  You can install it via *pip*:
138  
139  ```bash
140  pip install invoke
141  ```
142  
143  To setup invoke, you create a tasks.py file in the top directory of your project/package:
144  
145  
146  Here is an example *tasks.py* file:
147  
148  ```python
149  """Invoke build and compile tasks.
150  
151  """
152  import os
153  from urllib.request import pathname2url
154  from invoke import Collection, task
155  import webbrowser
156  
157  
158  def open_browser(file_name: str):
159      """Open the default browser at a local html file."""
160      uri = pathname2url(os.path.abspath(file_name))
161      webbrowser.open(f"file://{uri}")
162  
163  @task
164  def pip_install_requirements(c):
165      """Install dependencies needed to run packagename.
166  
167      Runs 'pip install -r requirements.txt'.
168      """
169      c.run('pip install -r requirements.txt')
170  
171  
172  @task(pip_install_requirements)
173  def pip_install_development_requirements(c):
174      """Install development dependencies needed to develop packagename.
175  
176      Runs 'pip install -r requirements_dev.txt'.
177      """
178      c.run('pip install -r requirements_dev.txt')
179  
180  
181  @task(pip_install_development_requirements)
182  def pip_install_emacs_requirements(c):
183      """Install all dependencies needed to develop with emacs as an IDE.
184  
185      Runs 'pip install -r requirements_emacs_dev.txt'.
186      """
187      c.run('pip install -r requirements_emacs_dev.txt')
188  
189  
190  @task(pip_install_development_requirements, default=True)
191  def pip_install_dev_mode(c):
192      """Install dev environment: required dev packages and also install locally in development mode.
193  
194      Runs 'pip install -e .'.
195      """
196      c.run('pip install -e .')
197  
198  
199  @task
200  def clean_release(c):
201      """Clean build artifacts produced when creating and releasing the package.
202  
203      Runs remove file and directory commands to remove sdist and python egg files and directories.
204      """
205      c.run('rm -rf build/')
206      c.run('rm -rf dist/')
207      c.run('rm -rf .eggs/')
208      c.run('rm -rf packagename.egg-info/')
209      c.run("find . -name '*.egg-info' -exec rm -fr {} +")
210      c.run("find . -name '*.egg' -exec rm -f {} +")
211  
212  
213  @task
214  def clean_pyc(c):
215      """Find and remove stale compiled python files ending in pyc or pyo as well as emacs backup files and __pycache__ directories."""
216      c.run("find . -name '*.pyc' -exec rm -f {} +")
217      c.run("find . -name '*.pyo' -exec rm -f {} +")
218      c.run("find . -name '*~' -exec rm -f {} +")
219      c.run("find . -name '__pycache__' -exec rm -fr {} +")
220  
221  
222  @task
223  def clean_testing(c):
224      """Remove cruft created during testing activities, including stale coverage and pytest output."""
225      c.run("rm -rf .tox/")
226      c.run("rm -f .coverage")
227      c.run("rm -rf htmlcov/")
228      c.run("rm -rf .pytest_cache")
229  
230  
231  @task
232  def clean_install(c):
233      """Uninstall packagename, regardless of original installation mode or method.
234  
235      Runs 'pip uninstall -y packagename'.
236      """
237      c.run("pip uninstall -y packagename")
238  
239  
240  @task(clean_release, clean_pyc, clean_testing, clean_install)
241  def pristine(c):
242      """Uninstall packagename and cleanup everything."""
243      pass
244  
245  
246  @task(default=True)
247  def unit_test(c):
248      """Run unit tests.
249  
250      Runs 'python -m unittest'.
251      """
252      c.run("python -m unittest")
253  
254  
255  @task
256  def coverage(c):
257      """Run unit tests and collect code coverage statistics.
258  
259      Runs:
260      'coverage run --source packagename -m unittest discover'
261      'coverage report -m'
262      'coverage html'
263      Then attempts to start a local browser pointed at ./htmlcov/index.html.
264      """
265      c.run("coverage run --source packagename -m unittest discover")
266      c.run("coverage report -m")
267      c.run("coverage html")
268      open_browser("./htmlcov/index.html")
269  
270  
271  @task
272  def lint_bin(c):
273      """Run pylint code quality static analyzer on the contents of the ./bin directory."""
274      c.run("pylint --fail-under=8 ./bin")
275  
276  
277  @task(lint_bin)
278  def lint(c):
279      """Run pylint code quality static analyzer on the contents of the ./packagename and ./bin directories."""
280      c.run("pylint --fail-under=8 ./packagename")
281  
282  
283  @task(lint)
284  def lint_tests(c):
285      """Run pylint code quality static analyzer on the contents of the ./packagename, ./bin and ./test directories."""
286      c.run("pylint --fail-under=2 ./tests")
287  
288  
289  @task
290  def security(c):
291      """Run bandit security static analyzer on the contents of the ./packagename directory."""
292      c.run("bandit -r packagename")
293  
294  
295  @task
296  def docs(c):
297      """Generate API documentation in markdown format and overlay ./docs directory.
298  
299      Done via pdoc3.
300      """
301      c.run("pdoc3 --pdf packagename > docs/README.md")
302  
303  
304  @task(docs, default=True)
305  def release(c):
306      """Build a package suitable for redistribution.
307  
308      Package can be uploaded to a PyPi compatible server such as an Azure DevOps Artifacts repository.
309  
310      Runs 'python setup.py sdist'
311      """
312      c.run("python setup.py sdist")
313  
314  
315  @task(docs)
316  def install_local(c):
317      """Install packagename locally via pip, see build.release and develop.install before running.
318  
319      This does not install in dev mode and is used mainly for testing. Use build.release to promote and develop.install to develop.
320  
321      Runs 'pip install .'.
322      """
323      c.run("pip install ./")
324  
325  ns = Collection()
326  
327  quality = Collection('quality')
328  quality.add_task(coverage)
329  quality.add_task(lint)
330  quality.add_task(lint_tests)
331  quality.add_task(security)
332  quality.add_task(unit_test)
333  
334  build = Collection('build')
335  build.add_task(clean_install, 'uninstall')
336  build.add_task(pristine)
337  build.add_task(install_local)
338  build.add_task(release)
339  
340  develop = Collection('develop')
341  develop.add_task(pip_install_emacs_requirements, 'install-emacs')
342  develop.add_task(pip_install_dev_mode, 'install')
343  
344  ns.add_collection(quality)
345  ns.add_collection(build)
346  ns.add_collection(develop)
347  
348  ```
349  
350  More info on *invoke*: <https://www.pyinvoke.org>