Commit Graph

2439 Commits

Author SHA1 Message Date
Maximilian Hils
3cb89069b9 fix compat with upcoming cryptography release 2021-10-05 08:19:37 +02:00
Maximilian Hils
4a3fefdf25 don't crash when sending content-length+transfer-encoding 2021-09-28 18:31:08 +02:00
Matthew Hughes
60a056a2d8 Don't set 'content-length' with 'transfer-encoding'
When updating the response content for a response, avoid adding the
'content-length' header if the response contains a 'transfer-encoding'
header, from the spec [1]:

> When a message does not have a Transfer-Encoding header field, a
Content-Length header field can provide the anticipated size, as a
decimal number of octets, for a potential payload body

Note the 'transfer-encoding' header is not used with HTTP/2

https://httpwg.org/specs/rfc7230.html#header.content-length
2021-09-28 18:31:08 +02:00
Maximilian Hils
7e24e77ac4
improve handling of flows with invalid content-lengths (#4819) 2021-09-23 18:03:52 +00:00
Matthew Hughes
7b4e219c4e
mitmweb: handle {en,de}coding on server-side (#4811)
* mitmweb: handle {en,de}coding on server-side

Handle this server-side rather than passing the message content encoding
details back when fetching flow content. If {en,de}coding fails, return
the raw request contents.

This addresses https://github.com/mitmproxy/mitmproxy/issues/4809

* fix typo

Co-authored-by: Maximilian Hils <github@maximilianhils.com>
2021-09-23 15:55:43 +02:00
Maximilian Hils
b41416b729
Merge pull request from GHSA-22gh-3r9q-xf38
This commit makes mitmproxy hard-fail when it encounters any attempts
at request/response smuggling.

For details, see https://github.com/mitmproxy/mitmproxy/security/advisories/GHSA-22gh-3r9q-xf38
2021-09-16 11:12:59 +02:00
Matthew Hughes
d9cac6fbcd
Make Cert.not{before,after} timezone aware (#4805)
I noticed when running tests the output of
`web/src/js/__tests__/ducks/_tflow.ts` would change depending on how I
set my timezone, e.g.

    $ TZ=America/Los_Angeles pytest --quiet \
        test/mitmproxy/tools/web/test_app.py >/dev/null \
        && grep --extended-regexp 'not(after|before)' web/src/js/__tests__/ducks/_tflow.ts
                "notafter": 2235132207,
                "notbefore": 1604415807,
    $ TZ=Asia/Tokyo pytest --quiet \
        test/mitmproxy/tools/web/test_app.py >/dev/null \
        && grep --extended-regexp 'not(after|before)'  web/src/js/__tests__/ducks/_tflow.ts
                "notafter": 2235074607,
                "notbefore": 1604354607

It looks like this is because the `cert_to_json` function simply calls
`timestamp` the `datetime` object from
`x509.Certificate.not_valid_before`, however, this `datetime` object is
not timestamp aware, from the docs [1]:

> A naïve datetime representing the beginning of the validity period for
the certificate in UTC

So when serializing to JSON, first convert the `datetime` to UTC then
call `timestamp`.

A test was added by inspecting one of the test certs with:

    $ openssl x509 -in test/mitmproxy/net/data/text_cert_2 -text

Extracting the date and asserting on that.

The corresponding test has also been re-run so that `_tflow.ts` was
regenerated with it's correct value. Snapshots were also updated via:

    $(npm bin)/jest --updateSnapshot

[1] https://cryptography.io/en/latest/x509/reference/#cryptography.x509.Certificate.not_valid_after
2021-09-16 10:53:36 +02:00
Maximilian Hils
0437d2935e make flake8 happy 2021-09-04 00:23:45 +02:00
Maximilian Hils
75ccd44c42 add more benchmark/memoryleak tooling 2021-09-03 21:34:53 +02:00
Maximilian Hils
d5bba9878b
Merge pull request #4780 from mhils/socks5-auth
Support SOCKS5 Authentication
2021-08-27 10:30:54 +02:00
Maximilian Hils
f9ffe8279d socks5: use correct version for auth negotiation 2021-08-27 10:25:31 +02:00
Maximilian Hils
f8826b29a2 cache socks5 auth for entire connection 2021-08-25 17:23:49 +02:00
Maximilian Hils
73d809a4c7 refactor proxyauth addon
the previous version was difficult to read,
this is hopefully better now.
2021-08-25 17:23:49 +02:00
Maximilian Hils
a3eca0b859 socks5 upstream auth: use proxyauth option 2021-08-25 17:23:49 +02:00
Maximilian Hils
c0fd6cfc09
Merge pull request #4779 from mhils/mitmweb-tcp
web: add support for viewing tcp flows
2021-08-25 13:35:37 +02:00
Maximilian Hils
550e1a4ab3
fix #3506 (#4777) 2021-08-23 19:35:31 +00:00
Maximilian Hils
fca21cbf72
GSoC Mitmweb Improvements (#4763)
* Add hover menu to FlowRow

* fix linting issue

* fix linting issue

* apply suggested changes

* add dropdown submenu

* apply requested change

* replace dropdown with popper.js-based implementation

* implement actions to  the menu

* clean FlowRow arguments

* web: major upgrades

This commit brings a bunch of under-the-hood mitmweb improvements:

 - migrate large parts of the codebase to typescript
 - introduce modern react testing conventions
 - vendor react-codemirror to silence warnings
 - use esbuild for both bundles and tests
 - move from yarn to npm
 - various fixes across the board

* web: fix options ducks

* add command bar to mitmweb

* fix app.py

* fix web test failure

* include command.js to commit

* fix optionsSpec.js

* transfer the argument suggestion into popup, show available commands, highlight currently typing argument

* fix lintings issue and test

* include package-lock.json

* add response to fetchMock

* update test

* update obsolete snapshot

* display help, description and add tab autocompletion

* change command history into mitmproxy's one

* move available commands popover to command help

* fix bug related to tab completion

* change CommandBar component into typescript, autoscroll overflowed result to bottom

* convert components in common and ContentView folder into typescript, and modified test

* fix bug of not dispatching when type in the field in FilterInput component

* make formatTimeStamp function return UTC time when updating snapshot in jest

* bind "this" to functions in CodeMirror

* convert components in FlowView, Header, Modal, ValueEditor into typescript

* applied requested changes

* [web] update fontawesome

* view: add id-based flow selection

* add raw `export` command

* contentviews: ignore empty data

* console: fix cert validity order

* major mitmweb upgrades

* fix tests

* fix web tests

* minor fixes

* please the coverage gods

* web: test coverage++, adjust commandbar

* web: update dependencies

* fix tests

* web: add backend tests

* web: convert everything to TypeScript

* web: fix HTTP trailer update

* tests++

Co-authored-by: zokutyou2@gmail.com <zokutyou2@gmail.com>
2021-08-23 11:54:54 +02:00
Maximilian Hils
d55aa7ea42 tests++ 2021-08-23 11:48:27 +02:00
Maximilian Hils
7219284d44 web: fix HTTP trailer update 2021-08-23 11:22:11 +02:00
Jinjie Zhang
0c366f6436 feat: add test script for socks5auth 2021-08-23 16:55:57 +08:00
Maximilian Hils
81c911345b
improve TLS version mismatch error, fix #4758 (#4772) 2021-08-23 07:15:56 +00:00
Maximilian Hils
03606fb0c0 fix tests 2021-08-20 18:49:44 +02:00
Maximilian Hils
2945ba925b please the coverage gods 2021-08-20 15:10:14 +02:00
Maximilian Hils
8561a620c8 minor fixes 2021-08-19 19:15:51 +02:00
Maximilian Hils
3ad4949c0e fix web tests 2021-08-19 19:15:51 +02:00
Maximilian Hils
0585e690c4 fix tests 2021-08-19 10:53:46 +02:00
Maximilian Hils
b320c6aa14 Merge remote-tracking branch 'origin/main' into gsoc 2021-08-18 17:39:01 +02:00
Maximilian Hils
65aa0a6ef5 major mitmweb upgrades 2021-08-18 17:38:22 +02:00
Maximilian Hils
f9b63e973e
Remove asyncio event loop workaround for tornado (#4762)
* remove asyncio event loop workaround for tornado

* Update CHANGELOG.md
2021-08-18 14:12:39 +00:00
Maximilian Hils
4fb3e4c321 treat encoding names case-insensitively, fix #4735
Co-authored-by: Mattwmaster58 <mattwmaster58@gmail.com>
2021-08-10 08:11:34 +02:00
Maximilian Hils
fb53df0cc1
expose full ClientHello object in tls_clienthello hook (#4744) 2021-08-09 15:41:05 +00:00
Maximilian Hils
fcec63eebc fix tnetstread crash found via fuzzing 2021-08-08 16:12:37 +02:00
Maximilian Hils
226e31290c Merge branch 'main' into gsoc 2021-08-04 19:16:31 +02:00
Maximilian Hils
4abd00afab
Merge pull request #4731 from aaron-tan/improve-render-size
Improve rendering of size column
2021-08-04 17:26:20 +02:00
Maximilian Hils
6aa9d8658f size column: add a bit more precision 2021-08-04 17:14:42 +02:00
Maximilian Hils
53a916e39b
fix websocket regression introduced in 7.0.1 (#4733) 2021-08-04 12:59:48 +00:00
Aaron Tan
4266c4852c Improve rendering of size column 2021-08-04 21:02:43 +10:00
Maximilian Hils
416e5046bc add proxy-auth header for plain http requests, fix #4728 2021-08-03 17:02:49 +02:00
Maximilian Hils
2ad3e5c698
Merge pull request #4716 from Zheaoli/fix-unittest
fix #4705
2021-08-03 14:43:18 +02:00
Maximilian Hils
04a1ff4a42 catch TypeError when reading flows, fix #4705 2021-08-03 14:34:16 +02:00
Maximilian Hils
083987c007
minor export fixups, fix #4718, fix #4714 (#4725) 2021-08-02 18:32:01 +00:00
Brad Dixon
68cc3e721f
Fix #4614 Options and Keybinding screen: home and end crash mitmproxy (#4693)
* Fix #4614 Options and Keybinding screen: home and end crash mitmproxy

* fix flake8

* Add integration test for fix to #4614

* fix the flake8, again.
2021-08-02 19:18:22 +02:00
Maximilian Hils
703fdea1a5
asgiapp: lowercase header names, refs #4717 (#4722) 2021-08-02 14:45:40 +00:00
Maximilian Hils
648a799525
don't fail on invalid certs, fix #4713 (#4721) 2021-08-02 13:15:05 +00:00
Maximilian Hils
b57bc68c51
Merge pull request #4710 from mhils/issue-4706
Improve HTTP Stream Modification
2021-08-02 14:36:44 +02:00
Maximilian Hils
51a15be50c extend test coverage for HTTP stream modification 2021-08-02 14:31:52 +02:00
Maximilian Hils
e865484c45
don't emit WS CONT. frames when the peer does not send any, fix #4701 (#4719) 2021-08-02 12:23:56 +00:00
Manjusaka
331061c6f2
add unittest for mitmproxy.io 2021-08-01 20:57:56 +08:00
Maximilian Hils
d8f5f0efbb
perf: reuse OpenSSL context to reduce number of TLS handshakes (#4694) 2021-07-21 09:23:27 +02:00
Maximilian Hils
7fe2c11898 tlsconfig: tests++ 2021-07-20 17:12:06 +02:00