We previously relied on the state of `Flow.reply` to check if a flow can be killed,
but this doesn't work anymore with `Flow.reply` being removed. Instead, we now
reintroduce the `Flow.live` attribute, which signals if we are on a live connection.
Killing still is not ideal (see comment in `Flow.killable`), but this paves the way.
The major, breaking change is that it is no longer possible to "take" a reply in
order to block the effect of a later addon hook.
This is patch 4/4 of the reply-ectomy.
In principle, a flow is killable as long as the connection handler is still
checking the error status of the flow.
This is patch 2/4 of the reply-ectomy.
This should improve behaviour, since calls to @concurrent will now be serialized
relative to other hooks on the same flow (but will still run in parallel with
hooks on different flows). Unlike a plain async hook, @concurrent allows blocking
sync APIs to run concurrently (e.g. requests).
This is patch 1/4 of the reply-ectomy.
Previously, it was throwing the following error:
File mitmproxy/addons/dumper.py, line 133, in _echo_request_line
elif flow.client_conn.peername:
AttributeError: 'bool' object has no attribute 'peername'
* Replays via upstream also need to comply with upstream handling
* Adjusted test for HTTP upstream replay which should NOT do a CONNECT
* Added Changelog
* Test for replay https pver upstream with CONNECT
* Proxy requests use full URL with host & port
* Finally remove some prints
* lint!
Co-authored-by: Maximilian Hils <github@maximilianhils.com>
* Replaced Kaitai protobuf parser with custom implementation, to prepare packed data handling
* Clamp varint size for bool conversion to 64bit, to satify tests
* moved WireParser into ProtoParser
* preserve work state
* Full packed support
* noc changes to example addon
* Adjusted test regex for ValueError
* Do not try to unpack fields twice
* Partial gRPC contentview prototype, not linted, no tests, not as add-on
* Linted (flake8)
* Save dev state
* Rewrote of protobuf parser, use decoding strategy, reduced rendered data. Parser uses generators
* minor cleanup
* fix: preferred encoding was provided as function instead of value
* flake8: line length
* Backlinked message tree objects, temporary debug out
* Partial implementation of gRPC definitions. Save state to fix a cras (data invalidate in edit mode)
* hack: deal with missing exception handling for generator based content views
* gRPC/Protoparser descriptions (with test code)
* replaced manual gzip decoding with mitmproxy.net.encoding.decode
* Refactored typing imports
* Reafctoring
* distinguish request vs response definitions, separate view config from parser config
* Code cleaning, moved customized protobuf definitions to example addon
* final cleanup
* changelog
* Stubs for tests
* Fixed render_riority of addon example
* Started adding tests
* Work on tests
* mypy
* Added pseudo encoder to tests, to cover special decodings
* Example addon test added
* finalized tests, no 100 percent coverage possible, see comments un uncovered code
* minor adjustments
* fixup tests
* Typos
Co-authored-by: Maximilian Hils <git@maximilianhils.com>
When updating the response content for a response, avoid adding the
'content-length' header if the response contains a 'transfer-encoding'
header, from the spec [1]:
> When a message does not have a Transfer-Encoding header field, a
Content-Length header field can provide the anticipated size, as a
decimal number of octets, for a potential payload body
Note the 'transfer-encoding' header is not used with HTTP/2
https://httpwg.org/specs/rfc7230.html#header.content-length
* mitmweb: handle {en,de}coding on server-side
Handle this server-side rather than passing the message content encoding
details back when fetching flow content. If {en,de}coding fails, return
the raw request contents.
This addresses https://github.com/mitmproxy/mitmproxy/issues/4809
* fix typo
Co-authored-by: Maximilian Hils <github@maximilianhils.com>
I noticed when running tests the output of
`web/src/js/__tests__/ducks/_tflow.ts` would change depending on how I
set my timezone, e.g.
$ TZ=America/Los_Angeles pytest --quiet \
test/mitmproxy/tools/web/test_app.py >/dev/null \
&& grep --extended-regexp 'not(after|before)' web/src/js/__tests__/ducks/_tflow.ts
"notafter": 2235132207,
"notbefore": 1604415807,
$ TZ=Asia/Tokyo pytest --quiet \
test/mitmproxy/tools/web/test_app.py >/dev/null \
&& grep --extended-regexp 'not(after|before)' web/src/js/__tests__/ducks/_tflow.ts
"notafter": 2235074607,
"notbefore": 1604354607
It looks like this is because the `cert_to_json` function simply calls
`timestamp` the `datetime` object from
`x509.Certificate.not_valid_before`, however, this `datetime` object is
not timestamp aware, from the docs [1]:
> A naïve datetime representing the beginning of the validity period for
the certificate in UTC
So when serializing to JSON, first convert the `datetime` to UTC then
call `timestamp`.
A test was added by inspecting one of the test certs with:
$ openssl x509 -in test/mitmproxy/net/data/text_cert_2 -text
Extracting the date and asserting on that.
The corresponding test has also been re-run so that `_tflow.ts` was
regenerated with it's correct value. Snapshots were also updated via:
$(npm bin)/jest --updateSnapshot
[1] https://cryptography.io/en/latest/x509/reference/#cryptography.x509.Certificate.not_valid_after
