Commit Graph

2488 Commits

Author SHA1 Message Date
Alexander Prinzhorn
8e1adbc5df
Add WebSocketMessage.injected flag (#5105)
* Add WebSocketMessage.injected flag

* add flow format migration

Co-authored-by: Maximilian Hils <git@maximilianhils.com>
2022-03-16 07:10:23 +00:00
Vinayak Khandelwal
3d5f6da048
ISSUE_5068 (#5161)
* changes for custom port number

* indent correction

* test coverage

* coverage correction

* simplify LDAP auth

* make mypy hapy

Co-authored-by: Maximilian Hils <git@maximilianhils.com>
2022-03-15 20:39:38 +00:00
Maximilian Hils
e22f8db526 add asyncio exception handler test 2022-03-15 19:59:51 +01:00
Maximilian Hils
290ec68aca remove pytest.mark.asyncio 2022-03-15 19:50:06 +01:00
Maximilian Hils
a7d473c1c1 fix compatibility with Python <3.10 2022-03-15 19:50:06 +01:00
Maximilian Hils
5fc20e3e8c tests++ 2022-03-15 18:43:56 +01:00
Maximilian Hils
b491528981 exclude time-sensitive test from CI 2022-03-04 11:38:28 +01:00
Maximilian Hils
372a632161 reintroduce Flow.live
We previously relied on the state of `Flow.reply` to check if a flow can be killed,
but this doesn't work anymore with `Flow.reply` being removed. Instead, we now
reintroduce the `Flow.live` attribute, which signals if we are on a live connection.
Killing still is not ideal (see comment in `Flow.killable`), but this paves the way.
2022-02-04 17:30:58 +01:00
Maximilian Hils
4f5930ec5d simplify concurrency tests 2022-02-04 17:30:20 +01:00
Robert Xiao
794c18bec0 Entirely remove the reply system.
The major, breaking change is that it is no longer possible to "take" a reply in
order to block the effect of a later addon hook.

This is patch 4/4 of the reply-ectomy.
2022-02-04 17:30:20 +01:00
Robert Xiao
ede269fce4 Flow.intercept: use an Event instead of the reply system
This is patch 3/4 of the reply-ectomy.
2022-02-04 17:30:20 +01:00
Robert Xiao
fd43ca19c4 Flow.kill: don't depend on reply status.
In principle, a flow is killable as long as the connection handler is still
checking the error status of the flow.

This is patch 2/4 of the reply-ectomy.
2022-02-04 17:30:20 +01:00
Robert Xiao
4448550746 Rewrite @concurrent to use async instead of the reply mechanism.
This should improve behaviour, since calls to @concurrent will now be serialized
relative to other hooks on the same flow (but will still run in parallel with
hooks on different flows). Unlike a plain async hook, @concurrent allows blocking
sync APIs to run concurrently (e.g. requests).

This is patch 1/4 of the reply-ectomy.
2022-02-04 17:30:20 +01:00
Robert Xiao
caf49300c2 Use async for tctx.cycle/tctx.invoke. 2022-02-04 17:30:20 +01:00
Robert Xiao
e186ccb3ba fix test.helper_tools.dumperview script.
Previously, it was throwing the following error:

  File mitmproxy/addons/dumper.py, line 133, in _echo_request_line
    elif flow.client_conn.peername:
AttributeError: 'bool' object has no attribute 'peername'
2022-02-04 17:30:20 +01:00
Robert Xiao
ee4999e8e4 Rename new async helper functions.
async_trigger -> trigger_event
invoke_addon -> invoke_addon_sync (API breakage)
async_invoke_addon -> invoke_addon
2022-02-04 17:30:20 +01:00
Robert Xiao
687ab4b589 Tests for async hook support 2022-02-04 17:30:20 +01:00
Maximilian Hils
8c86fd06db add changelog linkifier util 2022-02-04 16:00:44 +01:00
Maximilian Hils
53f60c88b1
fix a crash when refreshing headers with a negative unix timestamp, fix #5054 (#5078) 2022-01-18 22:32:38 +00:00
Maximilian Hils
d47fd3e9c3
ClientHello: add .raw_bytes(), refs #4877 (#5075) 2022-01-18 11:34:09 +00:00
Maximilian Hils
ef8c88da1f
setup websocket connection before finishing HTTP response, fix #5017 (#5020) 2021-12-27 15:58:24 +00:00
Maximilian Hils
3fbf3cf8ee tlsconfig: don't overwrite existing TLS context, refs #5019 2021-12-27 12:22:08 +01:00
Maximilian Hils
c74806feac
fix tests 2021-12-20 21:06:14 +01:00
Maximilian Hils
6997129bc0
make sure that running() is only invoked once on startup. (#4964)
fix #3584
2021-11-27 13:11:23 +00:00
Maximilian Hils
2dd845ed95 catch malformed cert warning in tests
we may need to catch this properly in `get_cert` at some point,
let's see if this ever turns out to be an issue.
2021-11-22 12:55:17 +01:00
Maximilian Hils
3cb87f5a2f split tls_handshake hook into client/server and success/fail variants 2021-11-22 10:23:21 +01:00
Maximilian Hils
a72f61ef57 Merge remote-tracking branch 'origin/main' into ignore-after-clienthello 2021-11-22 09:54:08 +01:00
shindexro
9a469806eb
quote argument of view.flows.resolve (#4910)
* Fix #4902

* Update type signature

* Switch to None check

* Fix spacing

* Quote view.flows.resolve argument

* Switch to call_strings
2021-11-19 13:04:20 +01:00
shindexro
888ce66f90
Correct flow-detail documentation (#4909)
* Fix #4902

* Update type signature

* Switch to None check

* Fix spacing

* Skip URL shortening when flow-detail is 0

* Sync docs with implementation

* Update URL shortening test
2021-11-18 08:26:05 +01:00
shindexro
39fa242e25
fix crash when invoking replay.server.count from console (#4905)
fix #4902
2021-11-17 16:53:43 +00:00
Peter Hoffmann
260fc68211
Fix #4876 Don't do CONNECT on plaintext HTTP replays via upstream (#4882)
* Replays via upstream also need to comply with upstream handling

* Adjusted test for HTTP upstream replay which should NOT do a CONNECT

* Added Changelog

* Test for replay https pver upstream with CONNECT

* Proxy requests use full URL with host & port

* Finally remove some prints

* lint!

Co-authored-by: Maximilian Hils <github@maximilianhils.com>
2021-10-31 20:23:04 +00:00
mame82
3a884ceb8f
gRPC packed repeated encoding (#4872)
* Replaced Kaitai protobuf parser with custom implementation, to prepare packed data handling

* Clamp varint size for bool conversion to 64bit, to satify tests

* moved WireParser into ProtoParser

* preserve work state

* Full packed support

* noc changes to example addon

* Adjusted test regex for ValueError

* Do not try to unpack fields twice
2021-10-21 13:20:13 +02:00
Maximilian Hils
79f464bc78
ignore superfluous newlines after HTTP CONNECT, fix #4870 (#4871) 2021-10-19 11:25:22 +00:00
mame82
8b6345e4a7
gRPC view: Adjusted parsing strategy for deprecated protobuf fields (#4858)
* Adjusted parsing strategy for deprecated protobuf fields

* Added test comment

* adjusted comment to reflect correct field IDs
2021-10-13 13:16:49 +02:00
mame82
480052f58b
Grpc contentview (#4851)
* Partial gRPC contentview prototype, not linted, no tests, not as add-on

* Linted (flake8)

* Save dev state

* Rewrote of protobuf parser, use decoding strategy, reduced rendered data. Parser uses  generators

* minor cleanup

* fix: preferred encoding was provided as function instead of value

* flake8: line length

* Backlinked message tree objects, temporary debug out

* Partial implementation of gRPC definitions. Save state to fix a cras (data invalidate in edit mode)

* hack: deal with missing exception handling for generator based content views

* gRPC/Protoparser descriptions (with test code)

* replaced manual gzip decoding with mitmproxy.net.encoding.decode

* Refactored typing imports

* Reafctoring

* distinguish request vs response definitions, separate view config from parser config

* Code cleaning, moved customized protobuf definitions to example addon

* final cleanup

* changelog

* Stubs for tests

* Fixed render_riority of addon example

* Started adding tests

* Work on tests

* mypy

* Added pseudo encoder to tests, to cover special decodings

* Example addon test added

* finalized tests, no 100 percent coverage possible, see comments un uncovered code

* minor adjustments

* fixup tests

* Typos

Co-authored-by: Maximilian Hils <git@maximilianhils.com>
2021-10-12 13:32:56 +02:00
Maximilian Hils
aad92c9d5a
Merge pull request #4847 from mhils/flowfilter
Flowfilter Improvements
2021-10-09 18:38:56 +02:00
Maximilian Hils
e3640801a5 lint! 2021-10-09 18:34:29 +02:00
Maximilian Hils
25bdf2f9d8 web: update filters 2021-10-08 18:39:22 +02:00
Maximilian Hils
c43a2ef8dc improve flowfilter api: raise on invalid input, add ~all 2021-10-08 17:45:47 +02:00
Maximilian Hils
adfccb90a5 minor fixes 2021-10-08 17:36:16 +02:00
Brad Dixon
9346002e0f
Add client_playback_concurrency option (#4842)
* nowait

* docs, tests, flake8

* we ideally support other values in the future

Co-authored-by: Maximilian Hils <github@maximilianhils.com>
2021-10-08 13:08:36 +02:00
Maximilian Hils
aa2f935dbb
Don't allow server.via change for live connections (#4841)
* don't allow `server.via` change for live connections

* return early if no tls context was set
2021-10-05 21:19:51 +02:00
Thomas Kriechbaumer
6be24f452e
Merge pull request #4810 from mhils/h2-fuzzing
Improve h2 Fuzzing Setup
2021-10-05 21:15:57 +02:00
Maximilian Hils
0f5b8c88af don't allow server.via change for live connections 2021-10-05 19:45:52 +02:00
Maximilian Hils
3cb89069b9 fix compat with upcoming cryptography release 2021-10-05 08:19:37 +02:00
Maximilian Hils
4a3fefdf25 don't crash when sending content-length+transfer-encoding 2021-09-28 18:31:08 +02:00
Matthew Hughes
60a056a2d8 Don't set 'content-length' with 'transfer-encoding'
When updating the response content for a response, avoid adding the
'content-length' header if the response contains a 'transfer-encoding'
header, from the spec [1]:

> When a message does not have a Transfer-Encoding header field, a
Content-Length header field can provide the anticipated size, as a
decimal number of octets, for a potential payload body

Note the 'transfer-encoding' header is not used with HTTP/2

https://httpwg.org/specs/rfc7230.html#header.content-length
2021-09-28 18:31:08 +02:00
Maximilian Hils
7e24e77ac4
improve handling of flows with invalid content-lengths (#4819) 2021-09-23 18:03:52 +00:00
Matthew Hughes
7b4e219c4e
mitmweb: handle {en,de}coding on server-side (#4811)
* mitmweb: handle {en,de}coding on server-side

Handle this server-side rather than passing the message content encoding
details back when fetching flow content. If {en,de}coding fails, return
the raw request contents.

This addresses https://github.com/mitmproxy/mitmproxy/issues/4809

* fix typo

Co-authored-by: Maximilian Hils <github@maximilianhils.com>
2021-09-23 15:55:43 +02:00
Maximilian Hils
a124b1eceb improve h2 fuzzing setup 2021-09-16 11:55:37 +02:00