Commit Graph

84 Commits

Author SHA1 Message Date
EndUser509
3a5550a09c
Rotate stream files (#5097)
* Example addon for saving streamed data including a small bug fix to make it work.

* Revert "Example addon for saving streamed data including a small bug fix to make it work."

This reverts commit 02ab78def9a52eaca1a89d0757cd9475ce250eaa.

* Add support for rotating stream files every hour or day

* Added tests

* Modified to change the stream file every time the formating string changes as time moves on.

* Update to more compact version

* simplify save addon logic

* make mypy happy

* fix compatibility with Python 3.8

Co-authored-by: Maximilian Hils <git@maximilianhils.com>
2022-03-17 10:41:05 +00:00
Maximilian Hils
e2f42ddb30
exit for all tools on startup error, fix #4544 (#5187) 2022-03-16 13:23:32 +00:00
Maximilian Hils
148429c0b3
lowercase user-added HTTP/2 headers, fix #4746 (#5186) 2022-03-16 10:59:30 +01:00
Maximilian Hils
6f0587734e
Refactor how we process --set options (#5067)
* refactor how we process `--set` options

* minor improvements based on @marwinxxii's review

* fix nits

* update changelog
2022-03-16 07:33:30 +00:00
Alexander Prinzhorn
8e1adbc5df
Add WebSocketMessage.injected flag (#5105)
* Add WebSocketMessage.injected flag

* add flow format migration

Co-authored-by: Maximilian Hils <git@maximilianhils.com>
2022-03-16 07:10:23 +00:00
Vinayak Khandelwal
3d5f6da048
ISSUE_5068 (#5161)
* changes for custom port number

* indent correction

* test coverage

* coverage correction

* simplify LDAP auth

* make mypy hapy

Co-authored-by: Maximilian Hils <git@maximilianhils.com>
2022-03-15 20:39:38 +00:00
EndUser509
a0cf273484
Merge pull request #5099 from EndUser509/save_streamed_data2
Save streamed data
2022-03-15 21:34:49 +01:00
Maximilian Hils
2387719ace urwid: support sgr mouse reporting
this commit vendors e2423b5069
so that we don't have to wait for a new release.
2022-03-13 19:16:01 +01:00
Maximilian Hils
65773cc9c0 speculative fix for #5158 2022-02-27 07:58:20 +01:00
Maximilian Hils
372a632161 reintroduce Flow.live
We previously relied on the state of `Flow.reply` to check if a flow can be killed,
but this doesn't work anymore with `Flow.reply` being removed. Instead, we now
reintroduce the `Flow.live` attribute, which signals if we are on a live connection.
Killing still is not ideal (see comment in `Flow.killable`), but this paves the way.
2022-02-04 17:30:58 +01:00
Robert Xiao
cee4b72459 Support async hooks. Fixes #4207. 2022-02-04 17:30:20 +01:00
Alexander Prinzhorn
437e55c2c2
await server_connected hook before doing something with the connection, fixes #5108 (#5110)
* await server_connected hook before doing something with the connection

* refine changelog wording

Co-authored-by: Maximilian Hils <github@maximilianhils.com>
2022-02-04 13:49:57 +00:00
pmoulton
b5c1ef11c1
Pass length of hostname.encode() to X509_VERIFY_PARAM_set1_host (#5083)
* Pass length of hostname.encode() to X509_VERIFY_PARAM_set1_host

Passing zero for the size_t length argument of
X509_VERIFY_PARAM_set1_host causes MITM Proxy to crash when used with
BoringSSL.

https://www.openssl.org/docs/man1.1.1/man3/X509_VERIFY_PARAM_set1_host.html

https://boringssl.googlesource.com/boringssl/

* Update CHANGELOG.md

Co-authored-by: Maximilian Hils <github@maximilianhils.com>
2022-01-21 12:22:01 +01:00
ian klatzco
1abb8f6921
add keyboard shortcut n to mitmweb (by using runCommand inside the kb shortcut handler) (#5061)
* add keyboard shortcut n to mitmweb (by creating /flows/create route)

* add keyboard shortcut n to mitmweb (simplified, by using runCommand instead of a new route)
2022-01-16 15:34:07 +01:00
Maximilian Hils
000e26674e
catch cancellation errors when draining writers, fix #5034 (#5040) 2022-01-08 08:06:58 +01:00
James Yale
ace07e7e3c
Example specified incorrect header (#4997)
* Example specified incorrect header

* Add CHANGELOG entry reference the documentation update

* fixup! Add CHANGELOG entry reference the documentation update
2021-12-20 20:18:00 +01:00
Maximilian Hils
6997129bc0
make sure that running() is only invoked once on startup. (#4964)
fix #3584
2021-11-27 13:11:23 +00:00
Brad Dixon
b1d0887db4
Add deepcopy to allow view.flows.resolve (issue #4916) (#4952)
* Add deepcopy to allow view.flows.resolve (issue #4916)

* try to simplify deepcopy implementation

* remove leftover check

Co-authored-by: Maximilian Hils <github@maximilianhils.com>
2021-11-26 23:14:06 +01:00
Maximilian Hils
3cb87f5a2f split tls_handshake hook into client/server and success/fail variants 2021-11-22 10:23:21 +01:00
Maximilian Hils
a72f61ef57 Merge remote-tracking branch 'origin/main' into ignore-after-clienthello 2021-11-22 09:54:08 +01:00
Marius
1c93a93696
Add font types to asset filter (~a) (#4928)
* Add font types to asset filter (~a)

* Add PR number to changelog

* remove flash mention

* restore asset test

Co-authored-by: Maximilian Hils <github@maximilianhils.com>
2021-11-21 15:47:09 +01:00
Karl Parkinson
df32d61086
Remove pyopenssl cruft (#4897)
* remove old pyopenssl cruft

* bump minimum version of pyopenssl

* add extra spaces to conform to style guide

* update changelog

* replace getattr with direct SSL method calls

* put version check back in but remove setdefault method calls

* tweak changelog wording

* bumb tox.ini pyOpenSSL dependency version

Co-authored-by: Karl Parkinson <karlparkinson@Karls-MBP.hitronhub.home>
2021-11-11 09:37:00 +01:00
Peter Hoffmann
260fc68211
Fix #4876 Don't do CONNECT on plaintext HTTP replays via upstream (#4882)
* Replays via upstream also need to comply with upstream handling

* Adjusted test for HTTP upstream replay which should NOT do a CONNECT

* Added Changelog

* Test for replay https pver upstream with CONNECT

* Proxy requests use full URL with host & port

* Finally remove some prints

* lint!

Co-authored-by: Maximilian Hils <github@maximilianhils.com>
2021-10-31 20:23:04 +00:00
Thomas Kriechbaumer
fffed0cb3a bump docker
closes #4846
2021-10-20 19:57:21 +02:00
Brad Dixon
77cf2ab4ee
fix for #4852 (#4857)
* fix for #4852

* changelog
2021-10-13 13:15:21 +02:00
mame82
480052f58b
Grpc contentview (#4851)
* Partial gRPC contentview prototype, not linted, no tests, not as add-on

* Linted (flake8)

* Save dev state

* Rewrote of protobuf parser, use decoding strategy, reduced rendered data. Parser uses  generators

* minor cleanup

* fix: preferred encoding was provided as function instead of value

* flake8: line length

* Backlinked message tree objects, temporary debug out

* Partial implementation of gRPC definitions. Save state to fix a cras (data invalidate in edit mode)

* hack: deal with missing exception handling for generator based content views

* gRPC/Protoparser descriptions (with test code)

* replaced manual gzip decoding with mitmproxy.net.encoding.decode

* Refactored typing imports

* Reafctoring

* distinguish request vs response definitions, separate view config from parser config

* Code cleaning, moved customized protobuf definitions to example addon

* final cleanup

* changelog

* Stubs for tests

* Fixed render_riority of addon example

* Started adding tests

* Work on tests

* mypy

* Added pseudo encoder to tests, to cover special decodings

* Example addon test added

* finalized tests, no 100 percent coverage possible, see comments un uncovered code

* minor adjustments

* fixup tests

* Typos

Co-authored-by: Maximilian Hils <git@maximilianhils.com>
2021-10-12 13:32:56 +02:00
Brad Dixon
9346002e0f
Add client_playback_concurrency option (#4842)
* nowait

* docs, tests, flake8

* we ideally support other values in the future

Co-authored-by: Maximilian Hils <github@maximilianhils.com>
2021-10-08 13:08:36 +02:00
Matthew Hughes
60a056a2d8 Don't set 'content-length' with 'transfer-encoding'
When updating the response content for a response, avoid adding the
'content-length' header if the response contains a 'transfer-encoding'
header, from the spec [1]:

> When a message does not have a Transfer-Encoding header field, a
Content-Length header field can provide the anticipated size, as a
decimal number of octets, for a potential payload body

Note the 'transfer-encoding' header is not used with HTTP/2

https://httpwg.org/specs/rfc7230.html#header.content-length
2021-09-28 18:31:08 +02:00
Maximilian Hils
7e24e77ac4
improve handling of flows with invalid content-lengths (#4819) 2021-09-23 18:03:52 +00:00
Maximilian Hils
eeb8a47806
add 7.0.3 changelog 2021-09-16 12:07:19 +02:00
Maximilian Hils
98a3e33477 tls: add tls_handshake, ignore-after-clienthello
this fixes #4702
2021-09-04 00:24:41 +02:00
Maximilian Hils
d5bba9878b
Merge pull request #4780 from mhils/socks5-auth
Support SOCKS5 Authentication
2021-08-27 10:30:54 +02:00
Maximilian Hils
a3eca0b859 socks5 upstream auth: use proxyauth option 2021-08-25 17:23:49 +02:00
Maximilian Hils
81c911345b
improve TLS version mismatch error, fix #4758 (#4772) 2021-08-23 07:15:56 +00:00
Maximilian Hils
d9d9a20ef2 tls: fix TLS1 constant
We accidentally reused the value for SSL3 here.
This is not as a bad as a it looks: First, neither version
is enabled by default. Second, because of how Python enums
work, this simply made the `TLS1` version unavailable
as an option (which is how I detected it).
2021-08-22 15:17:57 +02:00
Maximilian Hils
f9b63e973e
Remove asyncio event loop workaround for tornado (#4762)
* remove asyncio event loop workaround for tornado

* Update CHANGELOG.md
2021-08-18 14:12:39 +00:00
Maximilian Hils
4fb3e4c321 treat encoding names case-insensitively, fix #4735
Co-authored-by: Mattwmaster58 <mattwmaster58@gmail.com>
2021-08-10 08:11:34 +02:00
Maximilian Hils
8b88e8f0a5 mitmproxy 7.0.2 2021-08-04 15:01:35 +02:00
Maximilian Hils
45123cd287 update CHANGELOG 2021-08-03 17:14:07 +02:00
Maximilian Hils
d8f5f0efbb
perf: reuse OpenSSL context to reduce number of TLS handshakes (#4694) 2021-07-21 09:23:27 +02:00
Maximilian Hils
927f1d4ab3 update CHANGELOG 2021-07-20 15:33:05 +02:00
Maximilian Hils
ef2795673b disable HTTP/2 CONNECT for secure web proxies 2021-07-20 15:33:05 +02:00
Maximilian Hils
4511ea7c24 mitmproxy 7.0 2021-07-16 10:24:38 +02:00
Jesper Bränn
64961232e6
Make it possible to set sequence options (#4210)
* Make it possible to set sequence options

Attempts to fix #3015 through looking at whether or not the option is
of the type Sequence[str].

Treat all deferred options as potentially Sequence options, by making the
deferred dict values a list.

* Add full test coverage to optmanager again

* Document how to set sequence options

* minor improvements

* update changelog

Co-authored-by: Maximilian Hils <git@maximilianhils.com>
2021-06-23 18:08:24 +00:00
Maximilian Hils
34a620e57b
Docker: Add aarch64 Images (#4637)
* feat(cibuild): add buildx multi arch builds

* chore: add changelog for arm64

* temporarily enable docker ci job for PRs

* Update cibuild.py

* Update cibuild.py

* chore(cibuild): create docker-container xbuilder

* chore(cibuild): fix lint

* temporarily remove run check to see error message

* Update cibuild.py

* Update cibuild.py

* Update cibuild.py

* Update main.yml

* Update main.yml

* Update main.yml

* Update cibuild.py

* Update cibuild.py

* Update Dockerfile

* cleanup #1

* next test

* move to test branch

* fixup

* now upload

* enable armv6/7

* use multi-stage build to reduce image size

* armv7?

* drop armv6/armv7

Co-authored-by: Niels Hofmans <hello@ironpeak.be>
2021-06-15 13:47:50 +00:00
Maximilian Hils
fa6e8f1e9c [sans-io] add support for upstream_auth 2021-06-15 10:45:26 +02:00
Brad Dixon
4ee6bc79a0
Add json() method for HTTP Request and Response classes. (#4612)
* Add `json()` method for HTTP Request and Response classes.

* Raise errors when fetching content to decode as json.

* Update http.py

Co-authored-by: Maximilian Hils <github@maximilianhils.com>
2021-06-15 08:39:48 +00:00
Maximilian Hils
8e52c16b4c [sans-io] add support for body_size_limit 2021-06-13 15:56:33 +02:00
Maximilian Hils
199670cad4 move body streaming into proxy core, fix #4470 2021-06-13 15:56:33 +02:00
Roy Tu
bd00132b65
Fix multipart forms losing boundary values on edit (#4625)
* Fix for issue #4613

* Adding tests

* Updated CHANGELOG.md

* Restoring contentviews

* Reverting contentview tests

* Adding boundary generation and tests

* Extra newline for flake8

* Janky byte fix

* Revert "Extra newline for flake8"

This reverts commit 683ba167de2264d29f318e2bab83e13cbfb8812d.

* Reverting a commit that was supposed to go to dev branch

* Update CHANGELOG.md

* Update test_http.py

Co-authored-by: Maximilian Hils <github@maximilianhils.com>
2021-06-09 10:26:19 +00:00