Commit Graph

9691 Commits

Author SHA1 Message Date
Maximilian Hils
4a3fefdf25 don't crash when sending content-length+transfer-encoding 2021-09-28 18:31:08 +02:00
Matthew Hughes
60a056a2d8 Don't set 'content-length' with 'transfer-encoding'
When updating the response content for a response, avoid adding the
'content-length' header if the response contains a 'transfer-encoding'
header, from the spec [1]:

> When a message does not have a Transfer-Encoding header field, a
Content-Length header field can provide the anticipated size, as a
decimal number of octets, for a potential payload body

Note the 'transfer-encoding' header is not used with HTTP/2

https://httpwg.org/specs/rfc7230.html#header.content-length
2021-09-28 18:31:08 +02:00
Maximilian Hils
18d3144b39
[requires.io] dependency update (#4826)
Co-authored-by: requires.io <support@requires.io>
2021-09-27 17:43:43 +02:00
Ikko Ashimine
80bc6e147e
Fix typo in optmanager.py (#4825)
assigment -> assignment
2021-09-25 17:09:30 +02:00
Maximilian Hils
7e24e77ac4
improve handling of flows with invalid content-lengths (#4819) 2021-09-23 18:03:52 +00:00
Matthew Hughes
7b4e219c4e
mitmweb: handle {en,de}coding on server-side (#4811)
* mitmweb: handle {en,de}coding on server-side

Handle this server-side rather than passing the message content encoding
details back when fetching flow content. If {en,de}coding fails, return
the raw request contents.

This addresses https://github.com/mitmproxy/mitmproxy/issues/4809

* fix typo

Co-authored-by: Maximilian Hils <github@maximilianhils.com>
2021-09-23 15:55:43 +02:00
Maximilian Hils
c1003ee332
[requires.io] dependency update on main branch (#4814)
* [requires.io] dependency update

* [requires.io] dependency update

Co-authored-by: requires.io <support@requires.io>
2021-09-21 22:33:42 +02:00
Maximilian Hils
b5de2d75b5
ci: update ubuntu version used for builds 2021-09-21 22:31:58 +02:00
Maximilian Hils
667d4e0474 catch more tls errors where we know it's a trust issue 2021-09-16 13:38:37 +02:00
Maximilian Hils
eeb8a47806
add 7.0.3 changelog 2021-09-16 12:07:19 +02:00
Maximilian Hils
a124b1eceb improve h2 fuzzing setup 2021-09-16 11:55:37 +02:00
Maximilian Hils
cf32b18d2f
hyper-h2: catch IndexError as well 2021-09-16 11:41:52 +02:00
Maximilian Hils
b41416b729
Merge pull request from GHSA-22gh-3r9q-xf38
This commit makes mitmproxy hard-fail when it encounters any attempts
at request/response smuggling.

For details, see https://github.com/mitmproxy/mitmproxy/security/advisories/GHSA-22gh-3r9q-xf38
2021-09-16 11:12:59 +02:00
Matthew Hughes
d9cac6fbcd
Make Cert.not{before,after} timezone aware (#4805)
I noticed when running tests the output of
`web/src/js/__tests__/ducks/_tflow.ts` would change depending on how I
set my timezone, e.g.

    $ TZ=America/Los_Angeles pytest --quiet \
        test/mitmproxy/tools/web/test_app.py >/dev/null \
        && grep --extended-regexp 'not(after|before)' web/src/js/__tests__/ducks/_tflow.ts
                "notafter": 2235132207,
                "notbefore": 1604415807,
    $ TZ=Asia/Tokyo pytest --quiet \
        test/mitmproxy/tools/web/test_app.py >/dev/null \
        && grep --extended-regexp 'not(after|before)'  web/src/js/__tests__/ducks/_tflow.ts
                "notafter": 2235074607,
                "notbefore": 1604354607

It looks like this is because the `cert_to_json` function simply calls
`timestamp` the `datetime` object from
`x509.Certificate.not_valid_before`, however, this `datetime` object is
not timestamp aware, from the docs [1]:

> A naïve datetime representing the beginning of the validity period for
the certificate in UTC

So when serializing to JSON, first convert the `datetime` to UTC then
call `timestamp`.

A test was added by inspecting one of the test certs with:

    $ openssl x509 -in test/mitmproxy/net/data/text_cert_2 -text

Extracting the date and asserting on that.

The corresponding test has also been re-run so that `_tflow.ts` was
regenerated with it's correct value. Snapshots were also updated via:

    $(npm bin)/jest --updateSnapshot

[1] https://cryptography.io/en/latest/x509/reference/#cryptography.x509.Certificate.not_valid_after
2021-09-16 10:53:36 +02:00
Maximilian Hils
3bdc5ca0d1
protobuf: accept older versions 2021-09-16 10:50:15 +02:00
Maximilian Hils
38b0d27ad7
[requires.io] dependency update on main branch (#4807)
* [requires.io] dependency update

* [requires.io] dependency update

Co-authored-by: requires.io <support@requires.io>
2021-09-16 10:49:50 +02:00
Maximilian Hils
31f089cb8d
make urwid work with ProactorEventLoop (#4806) 2021-09-14 12:59:35 +00:00
Matthew Hughes
8c03f7bf8b
Fix warning in pytest (#4803)
Escape backslashes in pydocs containing '\*' (markdown escape sequence).
This removes the one warning I saw when running `pytest`:

    /mitmproxy/mitmproxy/http.py:97: DeprecationWarning: invalid escape sequence \*
2021-09-13 20:10:56 +00:00
Maximilian Hils
35596bd5e4
[requires.io] dependency update on main branch (#4801)
* [requires.io] dependency update

* [requires.io] dependency update

* [requires.io] dependency update

* Update setup.py

Co-authored-by: requires.io <support@requires.io>
2021-09-13 15:25:35 +00:00
Maximilian Hils
4229253a6e
bump installbuilder (#4800) 2021-09-13 15:02:15 +00:00
Maximilian Hils
e3556fbaad
Create SECURITY.md 2021-09-13 16:34:33 +02:00
Maximilian Hils
017344dfe4 tls: api docs++ 2021-09-04 17:03:26 +02:00
Maximilian Hils
7fd887a553 move tls hook data to mitmproxy.tls 2021-09-04 16:37:39 +02:00
Maximilian Hils
bdf4e31c58 move ClientHello to mitmproxy.tls 2021-09-04 16:10:39 +02:00
Maximilian Hils
9f39e2f387 tests++ 2021-09-04 16:03:06 +02:00
Maximilian Hils
98a3e33477 tls: add tls_handshake, ignore-after-clienthello
this fixes #4702
2021-09-04 00:24:41 +02:00
Maximilian Hils
0437d2935e make flake8 happy 2021-09-04 00:23:45 +02:00
Maximilian Hils
75ccd44c42 add more benchmark/memoryleak tooling 2021-09-03 21:34:53 +02:00
Maximilian Hils
4e5a0ae71d
charset detection: ignore case when searching in HTML (#4785) 2021-08-31 09:13:28 +00:00
Maximilian Hils
d5bba9878b
Merge pull request #4780 from mhils/socks5-auth
Support SOCKS5 Authentication
2021-08-27 10:30:54 +02:00
Maximilian Hils
f9ffe8279d socks5: use correct version for auth negotiation 2021-08-27 10:25:31 +02:00
Maximilian Hils
f8826b29a2 cache socks5 auth for entire connection 2021-08-25 17:23:49 +02:00
Maximilian Hils
73d809a4c7 refactor proxyauth addon
the previous version was difficult to read,
this is hopefully better now.
2021-08-25 17:23:49 +02:00
Maximilian Hils
a3eca0b859 socks5 upstream auth: use proxyauth option 2021-08-25 17:23:49 +02:00
Maximilian Hils
c0fd6cfc09
Merge pull request #4779 from mhils/mitmweb-tcp
web: add support for viewing tcp flows
2021-08-25 13:35:37 +02:00
Maximilian Hils
550e1a4ab3
fix #3506 (#4777) 2021-08-23 19:35:31 +00:00
Maximilian Hils
a6f673fb29
add python 3.10 trove classifier 2021-08-23 21:23:04 +02:00
Maximilian Hils
89beb31c8e web: make icon column sortable 2021-08-23 18:59:38 +02:00
Maximilian Hils
e483aa544f force less variable calculation 2021-08-23 14:58:56 +02:00
Maximilian Hils
fca21cbf72
GSoC Mitmweb Improvements (#4763)
* Add hover menu to FlowRow

* fix linting issue

* fix linting issue

* apply suggested changes

* add dropdown submenu

* apply requested change

* replace dropdown with popper.js-based implementation

* implement actions to  the menu

* clean FlowRow arguments

* web: major upgrades

This commit brings a bunch of under-the-hood mitmweb improvements:

 - migrate large parts of the codebase to typescript
 - introduce modern react testing conventions
 - vendor react-codemirror to silence warnings
 - use esbuild for both bundles and tests
 - move from yarn to npm
 - various fixes across the board

* web: fix options ducks

* add command bar to mitmweb

* fix app.py

* fix web test failure

* include command.js to commit

* fix optionsSpec.js

* transfer the argument suggestion into popup, show available commands, highlight currently typing argument

* fix lintings issue and test

* include package-lock.json

* add response to fetchMock

* update test

* update obsolete snapshot

* display help, description and add tab autocompletion

* change command history into mitmproxy's one

* move available commands popover to command help

* fix bug related to tab completion

* change CommandBar component into typescript, autoscroll overflowed result to bottom

* convert components in common and ContentView folder into typescript, and modified test

* fix bug of not dispatching when type in the field in FilterInput component

* make formatTimeStamp function return UTC time when updating snapshot in jest

* bind "this" to functions in CodeMirror

* convert components in FlowView, Header, Modal, ValueEditor into typescript

* applied requested changes

* [web] update fontawesome

* view: add id-based flow selection

* add raw `export` command

* contentviews: ignore empty data

* console: fix cert validity order

* major mitmweb upgrades

* fix tests

* fix web tests

* minor fixes

* please the coverage gods

* web: test coverage++, adjust commandbar

* web: update dependencies

* fix tests

* web: add backend tests

* web: convert everything to TypeScript

* web: fix HTTP trailer update

* tests++

Co-authored-by: zokutyou2@gmail.com <zokutyou2@gmail.com>
2021-08-23 11:54:54 +02:00
Maximilian Hils
d55aa7ea42 tests++ 2021-08-23 11:48:27 +02:00
Maximilian Hils
7219284d44 web: fix HTTP trailer update 2021-08-23 11:22:11 +02:00
Maximilian Hils
3589ec2f58 web: convert everything to TypeScript 2021-08-23 11:01:43 +02:00
Jinjie Zhang
0c366f6436 feat: add test script for socks5auth 2021-08-23 16:55:57 +08:00
Maximilian Hils
c5e3e3d636 web: add backend tests 2021-08-23 09:35:47 +02:00
Maximilian Hils
81c911345b
improve TLS version mismatch error, fix #4758 (#4772) 2021-08-23 07:15:56 +00:00
Aldo Cortesi
4f925848d9
[requires.io] dependency update on main branch (#4768)
* [requires.io] dependency update

* Update setup.py

Co-authored-by: requires.io <support@requires.io>
Co-authored-by: Maximilian Hils <git@maximilianhils.com>
2021-08-22 18:33:09 +02:00
Jinjie Zhang
5876e431ce fix: fix unused ctx import 2021-08-22 21:36:08 +08:00
Jinjie Zhang
9d31a66570 feat: add --socks5auth option to support socks5 userpassword authentication 2021-08-22 21:31:29 +08:00
Maximilian Hils
d9d9a20ef2 tls: fix TLS1 constant
We accidentally reused the value for SSL3 here.
This is not as a bad as a it looks: First, neither version
is enabled by default. Second, because of how Python enums
work, this simply made the `TLS1` version unavailable
as an option (which is how I detected it).
2021-08-22 15:17:57 +02:00