From 05a43572c87135b5e638fd2de765b1d600ccbe10 Mon Sep 17 00:00:00 2001 From: Maximilian Hils Date: Mon, 18 Jan 2021 13:56:27 +0100 Subject: [PATCH 1/2] make development setup instructions more clear Our dev.sh/dev.ps1 scripts feel like an unnecessary layer of abstraction. The revised docs make the install process transparent to experienced users, and can also be easily reused for other projects (hi, pdoc!). --- CONTRIBUTING.md | 76 ++++++++++++++++++++++++++----------------------- dev.ps1 | 15 ---------- dev.sh | 18 ------------ docs/README.md | 2 +- 4 files changed, 42 insertions(+), 69 deletions(-) delete mode 100644 dev.ps1 delete mode 100755 dev.sh diff --git a/CONTRIBUTING.md b/CONTRIBUTING.md index c87b23aff..6cd5e98e2 100644 --- a/CONTRIBUTING.md +++ b/CONTRIBUTING.md @@ -14,32 +14,49 @@ forward, please consider contributing in the following areas: ## Development Setup -To get started hacking on mitmproxy, please install a recent version of Python (we require at least Python 3.8). The -following commands should work on your system: +To get started hacking on mitmproxy, please install a recent version of Python (we require at least Python 3.8). +Then, do the following: +##### Linux / macOS ```shell +# 1) Verify that these commands work: python3 --version python3 -m pip --help python3 -m venv --help -``` - -If all of this run successfully, do the following: - -```shell +# 2) Install: git clone https://github.com/mitmproxy/mitmproxy.git cd mitmproxy -./dev.sh # "powershell .\dev.ps1" on Windows +python3 -m venv venv +venv/bin/pip install -e .[dev] ``` -The *dev* script will create a [virtualenv](https://virtualenv.pypa.io/) environment in a directory called "venv" and -install all mandatory and optional dependencies into it. The primary mitmproxy components are installed as "editable", -so any changes to the source in the repository will be reflected live in the virtualenv. +##### Windows +```shell +# 1) Verify that this command works: +python --version +# 2) Install: +git clone https://github.com/mitmproxy/mitmproxy.git +cd mitmproxy +python -m venv venv +venv\Scripts\pip install -e .[dev] +``` -The main executables for the project - `mitmdump`, `mitmproxy`, and `mitmweb` - are all created within the virtualenv. +This will clone mitmproxy's source code into a directory with the same name, +and then create an isolated Python environment (a [virtualenv](https://virtualenv.pypa.io/)) into which all dependencies are installed. +Mitmproxy itself is installed as "editable", so any changes to the source in the repository will be reflected live in the virtualenv. + +The main executables for the project – `mitmdump`, `mitmproxy`, and `mitmweb` – are all created within the virtualenv. After activating the virtualenv, they will be on your $PATH, and you can run them like any other command: +##### Linux / macOS ```shell -. venv/bin/activate # "venv\Scripts\activate" on Windows +source venv/bin/activate +mitmdump --version +``` + +##### Windows +```shell +venv\Scripts\activate mitmdump --version ``` @@ -52,41 +69,30 @@ basic test suite with [tox](https://tox.readthedocs.io/): tox -e py # runs Python tests ``` -Our CI system has additional tox environments that are run on every pull request and branch on GitHub. +Our CI system has additional tox environments that are run on every pull request (see [tox.ini](./tox.ini)). -For speedier testing, we recommend you run [pytest](http://pytest.org/) directly on individual test files or folders: +For speedier testing, you can also run [pytest](http://pytest.org/) directly on individual test files or folders: ```shell cd test/mitmproxy/addons pytest --cov mitmproxy.addons.anticache --cov-report term-missing --looponfail test_anticache.py ``` -Pytest does not check the code style, so you want to run `tox -e flake8` and `tox -e mypy` again before committing. - Please ensure that all patches are accompanied by matching changes in the test suite. The project tries to maintain 100% test coverage and enforces this strictly for some parts of the codebase. -## Documentation +### Code Style -The following tools are required to build the mitmproxy docs: +Keeping to a consistent code style throughout the project makes it easier to contribute and collaborate. -- [Hugo](https://gohugo.io/) (the extended version `hugo_extended` is required) -- [modd](https://github.com/cortesi/modd) - -```shell -cd docs -modd -``` - -## Code Style - -Keeping to a consistent code style throughout the project makes it easier to contribute and collaborate. Please stick to -the guidelines in [PEP8](https://www.python.org/dev/peps/pep-0008) unless there's a good reason not to. - -This is automatically enforced on every PR. If we detect a linting error, the PR checks will fail and block merging. You -can run our lint checks yourself with the following commands: +We enforce the following check for all PRs: ```shell tox -e flake8 -tox -e mypy # checks static types ``` + +If a linting error is detected, the automated pull request checks will fail and block merging. + +## Documentation + +Please check [docs/README.md](./docs/README.md) for instructions. diff --git a/dev.ps1 b/dev.ps1 deleted file mode 100644 index 3e776ab6d..000000000 --- a/dev.ps1 +++ /dev/null @@ -1,15 +0,0 @@ -$ErrorActionPreference = "Stop" - -python -m venv .\venv --copies -& .\venv\Scripts\activate.ps1 - -python -m pip install --disable-pip-version-check -U pip -cmd /c "pip install -r requirements.txt 2>&1" - -echo @" - - * Created virtualenv environment in .\venv. - * Installed all dependencies into the virtualenv. - * Activated virtualenv environment. - -"@ diff --git a/dev.sh b/dev.sh deleted file mode 100755 index df7b22d44..000000000 --- a/dev.sh +++ /dev/null @@ -1,18 +0,0 @@ -#!/usr/bin/env bash - -set -o errexit -set -o pipefail -set -o nounset -set -o xtrace - -echo "Creating dev environment in ./venv..." - -python3 -m venv venv -. venv/bin/activate -pip3 install -U pip setuptools -pip3 install -r requirements.txt - -echo "" -echo " * Created virtualenv environment in ./venv." -echo " * Installed all dependencies into the virtualenv." -echo " * You can now activate the $(python3 --version) virtualenv with this command: \`. venv/bin/activate\`" diff --git a/docs/README.md b/docs/README.md index 24c24d24c..cafba52e2 100644 --- a/docs/README.md +++ b/docs/README.md @@ -17,6 +17,6 @@ Now you can run `hugo server -D` in ./src. This is required to modify CSS files. - 1. Install hugo extended version. + 1. Install "extended" hugo version. You can now run `modd` in this directory instead of running hugo directly. From 494ccb5a22a0016152635b6eba853ca7b7c028e2 Mon Sep 17 00:00:00 2001 From: Maximilian Hils Date: Wed, 20 Jan 2021 15:06:02 +0100 Subject: [PATCH 2/2] remove requirements.txt --- requirements.txt | 1 - tox.ini | 8 +++----- 2 files changed, 3 insertions(+), 6 deletions(-) delete mode 100644 requirements.txt diff --git a/requirements.txt b/requirements.txt deleted file mode 100644 index aefbcb6d2..000000000 --- a/requirements.txt +++ /dev/null @@ -1 +0,0 @@ --e .[dev] diff --git a/tox.ini b/tox.ini index c88084e65..db488980a 100644 --- a/tox.ini +++ b/tox.ini @@ -5,7 +5,7 @@ toxworkdir={env:TOX_WORK_DIR:.tox} [testenv] deps = - -rrequirements.txt + -e .[dev] setenv = HOME = {envtmpdir} commands = mitmdump --version @@ -33,15 +33,13 @@ commands = mypy . {posargs} [testenv:individual_coverage] -deps = - -rrequirements.txt commands = python ./test/individual_coverage.py {posargs} [testenv:cibuild] passenv = CI_* GITHUB_* AWS_* TWINE_* DOCKER_* deps = - -rrequirements.txt + -e .[dev] pyinstaller==4.1 twine==3.3.0 awscli @@ -61,7 +59,7 @@ commands = [testenv:docs] passenv = GITHUB_* AWS_* deps = - -rrequirements.txt + -e .[dev] awscli changedir = docs commands =