Skip to content

ha-programming-python.org

Latest commit

 

History

History
447 lines (373 loc) · 16.7 KB

ha-programming-python.org

File metadata and controls

447 lines (373 loc) · 16.7 KB
Raw

Configuring Python in Emacs

import re

A literate programming file for configuring Python.

Introduction

The critical part of Python integration with Emacs is running LSP in Python using direnv. And the question to ask is if the Python we run it in Docker or in a virtual environment.

While Emacs supplies a Python editing environment, we’ll still use use-package to grab the latest:

(use-package python
  :after flycheck
  :mode (((rx ".flake8" eol) . conf-mode)
         ((rx "Pipfile" eol) . conf-mode)
         ((rx ".wsgi" eol)   . python-mode))
  :init
  (setq python-indent-guess-indent-offset-verbose nil
        flycheck-flake8-maximum-line-length 120)
  :config
  (setq python-shell-interpreter (or (executable-find "ipython") "python"))

  (flycheck-add-next-checker 'python-pylint 'python-pycompile 'append))

Keybindings

Instead of memorizing all the Emacs-specific keybindings, we use major-mode-hydra defined for python-mode:

(use-package major-mode-hydra
  :after python
  :config
  (defvar ha-python-eval-title (font-icons 'mdicon "run" :title "Python Evaluation"))
  (defvar ha-python-goto-title (font-icons 'faicon "python" :title "Python Symbol References"))
  (defvar ha-python-refactor-title (font-icons 'faicon "recycle" :title "Python Refactoring"))

  (pretty-hydra-define python-evaluate (:color blue :quit-key "C-g"
                                        :title ha-python-eval-title)
    ("Section"
     (("f" python-shell-send-defun "Function/class")
      ("e" python-shell-send-statement "Line")
      (";" python-shell-send-string "Expression"))
     "Entirety"
     (("f" python-shell-send-file "File")
      ("b" python-shell-send-buffer "Buffer")
      ("r" elpy-shell-send-region-or-buffer "Region"))))

  (pretty-hydra-define python-refactor (:color blue :quit-key "C-g"
                                        :title ha-python-refactor-title)
    ("Simple"
     (("r" iedit-mode "Rename"))
     "Imports"
     (("A" python-add-import "Add Import")
      ("a" python-import-symbol-at-point "Import Symbol")
      ("F" python-fix-imports "Fix Imports")
      ("S" python-sort-imports "Sort Imports"))))

  (pretty-hydra-define python-goto (:color pink :quit-key "C-g"
                                    :title ha-python-goto-title)
    ("Statements"
     (("s" xref-find-apropos "Find Symbol" :color blue)
      ("j" python-nav-forward-statement "Next")
      ("k" python-nav-backward-statement "Previous"))
     "Functions"
     (("F" imenu "Jump Function" :color blue)
      ("f" python-nav-forward-defun "Forward")
      ("d" python-nav-backward-defun "Backward")
      ("e" python-nav-end-of-defun "End of" :color blue))
     "Blocks"
     (("u" python-nav-up-list "Up" :color blue)
      (">" python-nav-forward-block "Forward")
      ("<" python-nav-backward-block "Backward"))))

  (major-mode-hydra-define python-mode (:quit-key "C-g" :color blue)
    ("Server"
     (("S" run-python "Start Server")
      ("s" python-shell-switch-to-shell "Go to Server"))
     "Edit"
     (("r" python-refactor/body "Refactor...")
      (">" python-indent-shift-left "Shift Left")
      ("<" python-indent-shift-right "Shift Right"))
     "Navigate/Eval"
     (("e" python-evaluate/body "Eval...")
      ("g" python-goto/body "Go to..."))
     "Docs"
     (("d" python-eldoc-at-point "Docs on Symbol")
      ("D" python-describe-at-point "Describe Symbol")))))

Sections below can add to this with major-mode-hydra-define+.

Note: Install the following packages globally for Emacs:

pip install flake8 flake8-bugbear pylint pyright mypy pycompile black ruff ipython

But certainly add those to each project’s requirements-dev.txt file.

iPython has a feature of loading code on startup per profile. First, create it with:

ipython profile create

Next, after reading David Vujic’s Are We There Yet essay, I took a look at his Python configuration, and added the auto reloading feature to the iPython profile configuration:

c = get_config()  #noqa

%load_ext autoreload
%autoreload 2

# c.InteractiveShellApp.extensions = ['autoreload']
# c.InteractiveShellApp.exec_lines = ['%autoreload 2']

Isolated Python Environments

While the Python community (and my work at my company) had difficulty transitioning from Python 2 to 3, I often run into issues needing a particular Python version and modules. After playing around with different approaches, I’m finding:

  • Docker environments are nicely isolated, but annoying to work from outside the container
  • The Builtin venv is works well for different library modules, but not for different versions
  • The pyenv deals with different Python versions, but is overkill for library isolation

While the auto-virtualenv project attempts to resolve this, I’m using the direnv project abstraction for situations where I need project-specific isolation in more than just Python.

Virtual Environments

Use the built-in module, venv, to create isolated Python environments for specific projects, enabling you to manage dependencies separately.

Create a virtual environment, either in the project’s directory, or in a global spot:

python3 -m venv .venv
#
python3 -m venv ~/.venv/my_project/

And then activate it:

source ~/.venv/my_project/bin/activate

Or add that to a projects’ .envrc.

Now, do what you need to do with this isolation:

pip install -r test-requirements.txt

Managing Python Versions

Pyenv is a tool for managing multiple versions of Python on your machine, allowing you to switch between them easily (see this essay). On a Mac, installed it via Homebrew:

brew install readline xz
brew install pyenv pyenv-virtualenv

Or on other systems, use the system Python to install pyenv globally:

pip install pyenv

Make sure we load this in the Zsh profile:

export PATH="$HOME/.pyenv/bin:$PATH"
eval "$(pyenv init -)"
eval "$(pyenv virtualenv-init -)"

Install the python versions you need, for instance:

pyenv install 3.9.23

Run pyenv versions to see what you have installed.

In any particular project directory, use a version you installed by creating a .python-version file, or call:

pyenv local 3.9.23

And have this in your .envrc file for use with direnv:

use python 3.7.1

Also, you need the following in your ~/.config/direnv/direnvrc file (which I have):

use_python() {
  local python_root=$(pyenv root)/versions/$1
  load_prefix "$python_root"
  if [[ -x "$python_root/bin/python" ]]; then
    layout python "$python_root/bin/python"
  else
    echo "Error: $python_root/bin/python can't be executed."
    exit
  fi
}

Tell Emacs about pyenv-mode:

(use-package pyenv-mode
  :config
  (defun setup-pyenv ()
    "Pyenv."
    (setenv "WORKON_HOME" "~/.pyenv/versions")
    (pyenv-mode +1)))

Now specify the pyenv Python version by calling pyenv-mode-set:

M-x pyenv-mode-set

When you run inferior Python processes (like run-python), the process will start inside the specified Python installation. You can unset the current version with:

M-x pyenv-mode-unset

Or, we can do it automatically when we get into a project (if the project has a .python-version file):

(use-package pyenv-mode
  :config
  (defun project-pyenv-mode-set (&rest _)
    "Set pyenv version matching project name."
    (ignore-errors
      (let* ((filename (thread-first
                         (project-current)
                         (project-root)
                         (file-name-concat ".python-version")))
             (version (when (file-exists-p filename)
                        (with-temp-buffer
                          (insert-file-contents filename)
                          (buffer-string)))))
        (when version
          (pyenv-mode-set version)
          (pyenv-mode-unset)))))

  ;; Either set/unset the pyenv version whenever changing tabs:
  (add-hook 'tab-bar-tab-post-select-functions 'project-pyenv-mode-set))

Docker Environment

Docker allows you to isolate your project’s environment. The downside is that you are using Docker and probably a bloated container. On my work laptop, a Mac, this creates a behemoth virtual machine that immediately spins the fans like a wind tunnel.

But, but… think of the dependencies!

Enough of the rant (I go back and forth), after getting Docker installed and running (ooo Podman … shiny), and you’ve created a Dockerfile for your project, let’s install container-env.

Your project’s .envrc file would contain something like:

CONTAINER_NAME=my-docker-container
CONTAINER_WRAPPERS=(python3 pip3 yamllint)
CONTAINER_EXTRA_ARGS="--env SOME_ENV_VAR=${SOME_ENV_VAR}"

container_layout

Editing Python Code

Let’s integrate this Python support for evil-text-object project:

(when (fboundp 'evil-define-text-object)
  (use-package evil-text-object-python
    :hook (python-mode . evil-text-object-python-add-bindings)))

This allows me to delete a Python “block” using dal.

Unit Tests

(use-package python-pytest
  :after python
  :commands python-pytest-dispatch
  :init
  (use-package major-mode-hydra
    :config
    (defvar ha-python-tests-title (font-icons 'devicon "pytest" :title "Python Test Framework"))
    (pretty-hydra-define python-tests (:color blue :quit-key "q"
                                              :title ha-python-tests-title)
      ("Suite"
       (("a" python-pytest "All")
        ("f" python-pytest-file-dwim "File DWIM")
        ("F" python-pytest-file "File"))
       "Specific"
       (("d" python-pytest-function-dwim "Function DWIM")
        ("D" python-pytest-function "Function"))
       "Again"
       (("r" python-pytest-repeat "Repeat tests")
        ("p" python-pytest-dispatch "Dispatch"))))

    (major-mode-hydra-define+ python-mode (:quit-key "q" :color blue)
      ("Misc"
       (("t" python-tests/body "Tests..."))))))

LSP Integration of Python

Dependencies

Each Python project’s requirements-dev.txt file would reference the python-lsp-server (not the unmaintained project, python-language-server):

python-lsp-server[all]

Note: This does mean, you would have a tox.ini with this line:

[tox]
minversion = 1.6
skipsdist = True
envlist = linters
ignore_basepython_conflict = True

[testenv]
basepython = python3
install_command = pip install {opts} {packages}
deps = -r{toxinidir}/test-requirements.txt
commands = stestr run {posargs}
           stestr slowest
# ...

Pyright

I’m using the Microsoft-supported pyright package instead, adding it to the global Python scope.

pyright

While the pyright package works with LSP, this isn’t needed, as eglot works directly with it.

eglot

The built-in Eglot package natively supports pyright via eglot-server-programs (no extra package needed):

(use-package eglot
  :straight nil
  :after python
  :defer t
  :hook (python-mode . eglot-ensure)
  :config
  (add-to-list 'eglot-server-programs
               '((python-mode python-ts-mode) . ("pyright-langserver" "--stdio"))))

Keybindings for the eglot-based LSP sub-menu, wired into the existing python-mode hydra:

(defvar ha-python-lsp-title (font-icons 'faicon "python" :title "Python LSP"))

(use-package eglot
  :straight nil
  :after major-mode-hydra
  :config
  (pretty-hydra-define python-lsp (:color blue :quit-key "q"
                                   :title ha-python-lsp-title)
    ("Server"
     (("D" eglot-shutdown "Disconnect")
      ("R" eglot-reconnect "Reconnect")
      ("?" eglot-show-workspace-configuration "Describe"))
     "Refactoring"
     (("a" eglot-code-actions "Code Actions")
      ("o" eglot-code-action-organize-imports "Organize Imports")
      ("r" eglot-rename "Rename"))
     "Navigation"
     (("." xref-find-definitions "Definition")
      ("/" xref-find-references "References")
      ("t" eglot-find-typeDefinition "Type Definition")
      ("i" eglot-find-implementation "Implementation"))
     "Docs"
     (("d" eldoc-doc-buffer "Eldoc Buffer")
      ("h" eglot-inlay-hints-mode "Inlay Hints")
      ("=" eglot-format-buffer "Format Buffer"))))

  (major-mode-hydra-define+ python-mode (:quit-key "q")
    ("Server"
     ( ;; ("L" eglot-ensure "Start eglot") --> Not needed with eglot-ensure
      ("l" python-lsp/body "LSP...")))))

Project Configuration

I work with a lot of projects with my team where I need to configure the project such that LSP and my Emacs setup works. Let’s suppose I could point a function at a project directory, and have it set it up:

(defun ha-python-configure-project (proj-directory)
  "Configure PROJ-DIRECTORY for LSP and Python."
  (interactive "DPython Project: ")

  (let ((default-directory proj-directory))
    (unless (f-exists? ".envrc")
      (message "Configuring direnv")
      (with-temp-file ".envrc"
        ;; (insert "use_python 3.7.4\n")
        (insert "layout_python3\n"))
      (direnv-allow))

    (unless (f-exists? ".pip.conf")
      (message "Configuring pip")
      (with-temp-file ".pip.conf"
        (insert "[global]\n")
        (insert "index-url = https://pypi.python.org/simple\n"))
      (shell-command "pipconf --local")
      (shell-command "pip install --upgrade pip"))

    (message "Configuring pip for eglot/pyright")
    (with-temp-file "requirements-dev.txt"
      (insert "pyright\n"))
    (shell-command "pip install -r requirements-dev.txt")))

Major Mode Hydra

Technical Artifacts

Let’s provide a name so we can require this file: