Commit Graph

58 Commits

Author SHA1 Message Date
mame82
480052f58b
Grpc contentview (#4851)
* Partial gRPC contentview prototype, not linted, no tests, not as add-on

* Linted (flake8)

* Save dev state

* Rewrote of protobuf parser, use decoding strategy, reduced rendered data. Parser uses  generators

* minor cleanup

* fix: preferred encoding was provided as function instead of value

* flake8: line length

* Backlinked message tree objects, temporary debug out

* Partial implementation of gRPC definitions. Save state to fix a cras (data invalidate in edit mode)

* hack: deal with missing exception handling for generator based content views

* gRPC/Protoparser descriptions (with test code)

* replaced manual gzip decoding with mitmproxy.net.encoding.decode

* Refactored typing imports

* Reafctoring

* distinguish request vs response definitions, separate view config from parser config

* Code cleaning, moved customized protobuf definitions to example addon

* final cleanup

* changelog

* Stubs for tests

* Fixed render_riority of addon example

* Started adding tests

* Work on tests

* mypy

* Added pseudo encoder to tests, to cover special decodings

* Example addon test added

* finalized tests, no 100 percent coverage possible, see comments un uncovered code

* minor adjustments

* fixup tests

* Typos

Co-authored-by: Maximilian Hils <git@maximilianhils.com>
2021-10-12 13:32:56 +02:00
Brad Dixon
9346002e0f
Add client_playback_concurrency option (#4842)
* nowait

* docs, tests, flake8

* we ideally support other values in the future

Co-authored-by: Maximilian Hils <github@maximilianhils.com>
2021-10-08 13:08:36 +02:00
Matthew Hughes
60a056a2d8 Don't set 'content-length' with 'transfer-encoding'
When updating the response content for a response, avoid adding the
'content-length' header if the response contains a 'transfer-encoding'
header, from the spec [1]:

> When a message does not have a Transfer-Encoding header field, a
Content-Length header field can provide the anticipated size, as a
decimal number of octets, for a potential payload body

Note the 'transfer-encoding' header is not used with HTTP/2

https://httpwg.org/specs/rfc7230.html#header.content-length
2021-09-28 18:31:08 +02:00
Maximilian Hils
7e24e77ac4
improve handling of flows with invalid content-lengths (#4819) 2021-09-23 18:03:52 +00:00
Maximilian Hils
eeb8a47806
add 7.0.3 changelog 2021-09-16 12:07:19 +02:00
Maximilian Hils
d5bba9878b
Merge pull request #4780 from mhils/socks5-auth
Support SOCKS5 Authentication
2021-08-27 10:30:54 +02:00
Maximilian Hils
a3eca0b859 socks5 upstream auth: use proxyauth option 2021-08-25 17:23:49 +02:00
Maximilian Hils
81c911345b
improve TLS version mismatch error, fix #4758 (#4772) 2021-08-23 07:15:56 +00:00
Maximilian Hils
d9d9a20ef2 tls: fix TLS1 constant
We accidentally reused the value for SSL3 here.
This is not as a bad as a it looks: First, neither version
is enabled by default. Second, because of how Python enums
work, this simply made the `TLS1` version unavailable
as an option (which is how I detected it).
2021-08-22 15:17:57 +02:00
Maximilian Hils
f9b63e973e
Remove asyncio event loop workaround for tornado (#4762)
* remove asyncio event loop workaround for tornado

* Update CHANGELOG.md
2021-08-18 14:12:39 +00:00
Maximilian Hils
4fb3e4c321 treat encoding names case-insensitively, fix #4735
Co-authored-by: Mattwmaster58 <mattwmaster58@gmail.com>
2021-08-10 08:11:34 +02:00
Maximilian Hils
8b88e8f0a5 mitmproxy 7.0.2 2021-08-04 15:01:35 +02:00
Maximilian Hils
45123cd287 update CHANGELOG 2021-08-03 17:14:07 +02:00
Maximilian Hils
d8f5f0efbb
perf: reuse OpenSSL context to reduce number of TLS handshakes (#4694) 2021-07-21 09:23:27 +02:00
Maximilian Hils
927f1d4ab3 update CHANGELOG 2021-07-20 15:33:05 +02:00
Maximilian Hils
ef2795673b disable HTTP/2 CONNECT for secure web proxies 2021-07-20 15:33:05 +02:00
Maximilian Hils
4511ea7c24 mitmproxy 7.0 2021-07-16 10:24:38 +02:00
Jesper Bränn
64961232e6
Make it possible to set sequence options (#4210)
* Make it possible to set sequence options

Attempts to fix #3015 through looking at whether or not the option is
of the type Sequence[str].

Treat all deferred options as potentially Sequence options, by making the
deferred dict values a list.

* Add full test coverage to optmanager again

* Document how to set sequence options

* minor improvements

* update changelog

Co-authored-by: Maximilian Hils <git@maximilianhils.com>
2021-06-23 18:08:24 +00:00
Maximilian Hils
34a620e57b
Docker: Add aarch64 Images (#4637)
* feat(cibuild): add buildx multi arch builds

* chore: add changelog for arm64

* temporarily enable docker ci job for PRs

* Update cibuild.py

* Update cibuild.py

* chore(cibuild): create docker-container xbuilder

* chore(cibuild): fix lint

* temporarily remove run check to see error message

* Update cibuild.py

* Update cibuild.py

* Update cibuild.py

* Update main.yml

* Update main.yml

* Update main.yml

* Update cibuild.py

* Update cibuild.py

* Update Dockerfile

* cleanup #1

* next test

* move to test branch

* fixup

* now upload

* enable armv6/7

* use multi-stage build to reduce image size

* armv7?

* drop armv6/armv7

Co-authored-by: Niels Hofmans <hello@ironpeak.be>
2021-06-15 13:47:50 +00:00
Maximilian Hils
fa6e8f1e9c [sans-io] add support for upstream_auth 2021-06-15 10:45:26 +02:00
Brad Dixon
4ee6bc79a0
Add json() method for HTTP Request and Response classes. (#4612)
* Add `json()` method for HTTP Request and Response classes.

* Raise errors when fetching content to decode as json.

* Update http.py

Co-authored-by: Maximilian Hils <github@maximilianhils.com>
2021-06-15 08:39:48 +00:00
Maximilian Hils
8e52c16b4c [sans-io] add support for body_size_limit 2021-06-13 15:56:33 +02:00
Maximilian Hils
199670cad4 move body streaming into proxy core, fix #4470 2021-06-13 15:56:33 +02:00
Roy Tu
bd00132b65
Fix multipart forms losing boundary values on edit (#4625)
* Fix for issue #4613

* Adding tests

* Updated CHANGELOG.md

* Restoring contentviews

* Reverting contentview tests

* Adding boundary generation and tests

* Extra newline for flake8

* Janky byte fix

* Revert "Extra newline for flake8"

This reverts commit 683ba167de2264d29f318e2bab83e13cbfb8812d.

* Reverting a commit that was supposed to go to dev branch

* Update CHANGELOG.md

* Update test_http.py

Co-authored-by: Maximilian Hils <github@maximilianhils.com>
2021-06-09 10:26:19 +00:00
Brad Dixon
6d2b823a54
Add flow.comment command and keybinding to add a comment to a flow. (#4608)
* Add `flow.comment` command and keybinding to add a comment to a flow.

* Store comment in Flow().comment. Add ~comment flowfilter syntax.

* resolve: Pythonic flow.comment

* Be consistent and use comment variable.
2021-05-28 20:38:27 +02:00
Alexander Prinzhorn
4f60e52413
Fix parsing of certificate issuer/subject with escaped special chars (#4557)
* keyinfo typing

* Fix parsing of certificate issuer/subject with escaped special characters

* tests

* rfc4514_attribute_name and multi value test

* pyca version + mypy happy dance

* aT lEaSt tTO sPAceS BeFOre iNLinE cOMment

* fix coverage

Co-authored-by: Maximilian Hils <github@maximilianhils.com>
2021-05-27 09:51:01 +00:00
Brad Dixon
c6ba97eab6
Use emoji and characters as markers. Add ~marker filter syntax. (#4587)
* Use emoji and characters as markers. Add ~marker filter syntax.

* Add a test to please our CI overlords. :)
2021-05-27 11:40:41 +02:00
Niels Hofmans
5a03098d23
Fix: check pending writes before requesting more from server (#4564)
* fix(server): drain writer after writing

* fix(server): move writer drain

* chore(changelog): add ref

* fixup changelog

* fix race condition in flow control code

Co-authored-by: Maximilian Hils <git@maximilianhils.com>
2021-05-14 09:24:19 +00:00
Brad Dixon
da07cb78a6 Add metadata filter syntax: ~meta 2021-05-11 09:12:47 -04:00
Ben
df9dc4892b
Add spacing to improve readability of fingerprint (#4588) 2021-05-09 22:54:10 +02:00
Brad Dixon
8d8e10672e
Allow browser.start to open multiple browsers. (#4585) 2021-05-08 22:13:15 +02:00
Maximilian Hils
1b0fce6aba Revert "Copy examples into Docker image (#4447)"
This reverts commit a5ed1d377b.
2021-04-05 10:31:46 +02:00
Anatoli Babenia
a5ed1d377b
Copy examples into Docker image (#4447) 2021-04-05 10:20:52 +02:00
Peter Dave Hello
f1b55fc817
Fix typo: Github should be GitHub (#4541) 2021-04-05 10:08:21 +02:00
Maximilian Hils
68c55979fb update CHANGELOG.md 2021-03-30 11:23:50 +02:00
Maximilian Hils
9baaf9dfdb
minor cleanups (#4530) 2021-03-29 17:42:24 +02:00
HereC
de3f089bb0
Add block_list Option to set HTTP status for blocked flows/urls (#4516)
* Add block_list Option which returns a set HTTP status for a blocked flow filter:URL

* Add changelog entry

* Add blocklist to default AddOns

* Add Nginx 444 status code to list, so it is available for validation

* Add overview of blocklists

* Add allow-only, and more tests

* Fix mypy tox issue

* Finish test coverage with test for invalid filter

* Add PR feedback

* Fix type/scope error

* Fix stray blank line

* Delete concepts-blocklist.md

* Feature in overview

* Add flag to metadata for blocklisted flows.

* minor improvements

Co-authored-by: Maximilian Hils <git@maximilianhils.com>
2021-03-29 15:30:21 +00:00
Peter Dave Hello
123342ea0b
Improve Markdown syntax(styling) (#4496)
* Improve Markdown syntax

- Add missing blank lines
- Remove trailing spaces
- Remove additional blank lines
- Fix indentation consistency and correctness

* Update addons-examples.md

Co-authored-by: Maximilian Hils <github@maximilianhils.com>
2021-03-11 19:13:02 +01:00
Peter Dave Hello
27ba85bc30
Switch Docker release to be based on Debian instead of Alpine (#4493) 2021-03-11 16:49:07 +01:00
Maximilian Hils
70e08c880c
don't reuse closed connections, refs #4451 (#4458) 2021-02-18 22:07:18 +00:00
Denis Kasak
856a35af6d
Use original flow host instead of IP when exporting to curl/httpie. (#4307)
Use original flow host instead of IP when exporting to curl/httpie.

Unless this is done, the SNI server name will not be sent, often making
the curl/httpie command have different behaviour than the original
request (most often in the form of failing to establish a TLS
connection).

With this change, we always use the original host, fixing this failure.
However, if the original host is a domain, it may sometimes resolve to
a different IP address later on. In curl, we solve this problem by
forcing it to connect to the original IP using `--resolve`. For httpie
there is currently no easy solution (see:
https://github.com/httpie/httpie/issues/414).
2021-02-09 18:44:46 +00:00
Alexander Prinzhorn
4212a56f25
Collect all flow filter matches before modifying headers, fixes #4245 (#4246) 2021-02-09 19:37:46 +01:00
Anatoli Babenia
578514d4f2
Use ? to exit from help if already there (#4420)
* Use `?` to exit from help if already there

* Add `?` help exit shortcut to CHANGELOG
2021-02-03 00:15:42 +01:00
Graham Robbins
2694b05fe9
Extract port number from authority before IDNA decode (#4410)
* Extract port from authority before IDNA decode

A UnicodeError exception may be raised if the port is present

* Update Changelog

* Test for badly formed byte input
2021-01-24 20:45:01 +01:00
Kamil Borzym
f51019cb63
[#4235] Automatic view mode based on should_render method (#4236)
* [#4235] Automatic view based on should_render method instead of content_types property

* [#4235] Update CHENGELOG

* [#4235] Fix linter warnings

* Add an explicit test for the new forward-compatible behaviour

* wip

* contentviews: introduce render_priority (2/2)

* coverage++, lint!

* minor fixes

Co-authored-by: Maximilian Hils <git@maximilianhils.com>
2021-01-17 11:39:51 +01:00
Maximilian Hils
667cacba3f generate event documentation from source 2021-01-05 21:54:44 +01:00
Maximilian Hils
ca45548289
Merge pull request #4384 from mhils/mitmproxy.exe
mitmproxy.exe 🎉🥳
2021-01-05 08:18:11 +01:00
Maximilian Hils
605961c57f build mitmproxy.exe 🎉 2021-01-05 07:46:53 +01:00
Maximilian Hils
455fee1126 warn about nonfunctioning options, permanently remove unused ones.
If you are affected by this change, please do reach out by filing an issue.
2021-01-01 18:34:15 +01:00
Maximilian Hils
0bbb5ca345 reverse proxy: do not assume TLS for non-standard ports 2021-01-01 16:57:50 +01:00