Grasscutters/mitmproxy
3
0
Fork 0
mirror of https://github.com/Grasscutters/mitmproxy.git synced 2024-12-01 11:41:54 +00:00
Code Issues Projects Releases Wiki Activity
9,598 Commits 1 Branch 67 Tags 50 MiB
95089486ca
Commit Graph

2463 Commits

Author SHA1 Message Date
Maximilian Hils
1858564b91 add WebSocketMessage.text 2021-06-22 17:39:55 +02:00
Maximilian Hils
9b119c3dac web: major upgrades 
This commit brings a bunch of under-the-hood mitmweb improvements:

 - migrate large parts of the codebase to typescript
 - introduce modern react testing conventions
 - vendor react-codemirror to silence warnings
 - use esbuild for both bundles and tests
 - move from yarn to npm
 - various fixes across the board
 2021-06-21 17:51:47 +02:00
Maximilian Hils
2fe4a49d8f logging: don't pick up later mutations 2021-06-20 02:15:09 +02:00
Maximilian Hils
d6975e0b80 add next_layer docs + example, minor fixes 2021-06-20 02:15:09 +02:00
Maximilian Hils
159c064fb4 enable addons to hardcode client.alpn 2021-06-19 12:29:18 +02:00
Maximilian Hils
6178b4b72a raise if server address is updated on non-closed connections 2021-06-19 12:28:40 +02:00
Maximilian Hils
fb625c6868 fix h2 bug found via fuzzing 2021-06-15 10:45:26 +02:00
Maximilian Hils
76f0b8f65e fix tests 2021-06-15 10:45:26 +02:00
Maximilian Hils
fa6e8f1e9c [sans-io] add support for upstream_auth 2021-06-15 10:45:26 +02:00
Brad Dixon
4ee6bc79a0
Add json() method for HTTP Request and Response classes. (#4612) 
* Add `json()` method for HTTP Request and Response classes.

* Raise errors when fetching content to decode as json.

* Update http.py

Co-authored-by: Maximilian Hils <github@maximilianhils.com>
 2021-06-15 08:39:48 +00:00
Maximilian Hils
1b13411383 update ruamel.yaml, adjust api 2021-06-13 19:59:11 +02:00
Maximilian Hils
8e52c16b4c [sans-io] add support for body_size_limit 2021-06-13 15:56:33 +02:00
Maximilian Hils
199670cad4 move body streaming into proxy core, fix #4470 2021-06-13 15:56:33 +02:00
Maximilian Hils
83a46b13b9 further simplify ALPN selection 2021-06-10 00:18:37 +02:00
Maximilian Hils
d47eb7556a fix #4630 2021-06-09 23:59:44 +02:00
Roy Tu
bd00132b65
Fix multipart forms losing boundary values on edit (#4625) 
* Fix for issue #4613

* Adding tests

* Updated CHANGELOG.md

* Restoring contentviews

* Reverting contentview tests

* Adding boundary generation and tests

* Extra newline for flake8

* Janky byte fix

* Revert "Extra newline for flake8"

This reverts commit 683ba167de2264d29f318e2bab83e13cbfb8812d.

* Reverting a commit that was supposed to go to dev branch

* Update CHANGELOG.md

* Update test_http.py

Co-authored-by: Maximilian Hils <github@maximilianhils.com>
 2021-06-09 10:26:19 +00:00
Brad Dixon
6d2b823a54
Add flow.comment command and keybinding to add a comment to a flow. (#4608) 
* Add `flow.comment` command and keybinding to add a comment to a flow.

* Store comment in Flow().comment. Add ~comment flowfilter syntax.

* resolve: Pythonic flow.comment

* Be consistent and use comment variable.
 2021-05-28 20:38:27 +02:00
Alexander Prinzhorn
4f60e52413
Fix parsing of certificate issuer/subject with escaped special chars (#4557) 
* keyinfo typing

* Fix parsing of certificate issuer/subject with escaped special characters

* tests

* rfc4514_attribute_name and multi value test

* pyca version + mypy happy dance

* aT lEaSt tTO sPAceS BeFOre iNLinE cOMment

* fix coverage

Co-authored-by: Maximilian Hils <github@maximilianhils.com>
 2021-05-27 09:51:01 +00:00
Brad Dixon
c6ba97eab6
Use emoji and characters as markers. Add ~marker filter syntax. (#4587) 
* Use emoji and characters as markers. Add ~marker filter syntax.

* Add a test to please our CI overlords. :)
 2021-05-27 11:40:41 +02:00
Maximilian Hils
b70358cbde
Merge pull request #4586 from rbdixon/meta_replay_filter 
Metadata and replay filter syntax
 2021-05-14 08:37:39 +02:00
Maximilian Hils
09bd608174
fix #4551 (#4594) 2021-05-11 16:03:39 +02:00
Brad Dixon
da07cb78a6 Add metadata filter syntax: ~meta 2021-05-11 09:12:47 -04:00
Brad Dixon
69f555f9bb Add replay filter syntax: ~replay, ~replayq, ~replays 2021-05-11 09:09:10 -04:00
Brad Dixon
8d8e10672e
Allow browser.start to open multiple browsers. (#4585) 2021-05-08 22:13:15 +02:00
Maximilian Hils
0ca458fd64
Merge pull request #4535 from mhils/master-main 
master branch -> main branch
 2021-03-30 17:03:14 +02:00
Maximilian Hils
ead33008bc master branch -> main branch 2021-03-30 15:30:45 +02:00
Maximilian Hils
eefc55f28f coverage++ 2021-03-30 13:03:41 +02:00
Maximilian Hils
f94a9a3c9d
Merge pull request #4529 from mhils/eager-sans-io 
Sans-IO Improvements: Connection Strategy
 2021-03-30 10:16:57 +02:00
Maximilian Hils
338cd0b00a
Merge pull request #4531 from mhils/selfconnect 
Detect recursive self-connects and stop them
 2021-03-30 09:39:27 +02:00
Maximilian Hils
945e4e3ef5 detect recursive self-connects and stop them (attempt 2) 2021-03-30 09:25:35 +02:00
Maximilian Hils
be20765129 detect recursive self-connects and stop them 2021-03-29 22:38:11 +02:00
HereC
de3f089bb0
Add block_list Option to set HTTP status for blocked flows/urls (#4516) 
* Add block_list Option which returns a set HTTP status for a blocked flow filter:URL

* Add changelog entry

* Add blocklist to default AddOns

* Add Nginx 444 status code to list, so it is available for validation

* Add overview of blocklists

* Add allow-only, and more tests

* Fix mypy tox issue

* Finish test coverage with test for invalid filter

* Add PR feedback

* Fix type/scope error

* Fix stray blank line

* Delete concepts-blocklist.md

* Feature in overview

* Add flag to metadata for blocklisted flows.

* minor improvements

Co-authored-by: Maximilian Hils <git@maximilianhils.com>
 2021-03-29 15:30:21 +00:00
Maximilian Hils
63e31de14e reuse certificate errors to avoid unnecessary connections 2021-03-29 15:07:26 +02:00
Maximilian Hils
d3f3725479 wait for TLS ClientHello when connection_strategy=eager 2021-03-29 15:07:26 +02:00
Maximilian Hils
aa815a5246 make connection_strategy=eager the default 2021-03-29 15:07:26 +02:00
Maximilian Hils
89603e5e6f proxy: properly catch ZeroReturnError 2021-03-29 15:07:26 +02:00
Maximilian Hils
80f208fb2a tlsconfig: don't negotiate ALPN with client if server refused to do so 2021-03-29 15:07:26 +02:00
Maximilian Hils
0650f132e9
Merge pull request #4502 from mhils/inject 
Add WebSocket/TCP Message Injection
 2021-03-16 15:07:06 +01:00
Maximilian Hils
fe6f0d368b inject -> inject.tcp/inject.websocket 2021-03-16 15:01:33 +01:00
Maximilian Hils
7fae3c8951 code style improvements 2021-03-15 17:24:52 +01:00
Maximilian Hils
12e4785d44 use request/response hooks for trailers 2021-03-15 17:12:10 +01:00
sanlengjingvv
ed7067d36d clean code, 
support request trailers when response body is streamed,
support  trailers when  body is streamed
 2021-03-15 11:30:20 +08:00
sanlengjingvv
a3fd70f240 fix lint error 2021-03-14 18:42:29 +08:00
sanlengjingvv
76d7ee3a2f support http/2 response trailers 2021-03-14 18:31:14 +08:00
Maximilian Hils
c842abff2d coverage++ 2021-03-13 23:17:07 +01:00
Maximilian Hils
13db172320 coverage++ 2021-03-13 14:47:42 +01:00
Maximilian Hils
c331d2eeb2 add TCP message injection 2021-03-12 21:15:07 +01:00
Maximilian Hils
5921c590e3 injection: ConnectionEvent -> Event 2021-03-12 16:42:55 +01:00
Maximilian Hils
07f1bcf543 add WebSocket inject command 2021-03-12 16:28:21 +01:00
Peter Dave Hello
123342ea0b
Improve Markdown syntax(styling) (#4496) 
* Improve Markdown syntax

- Add missing blank lines
- Remove trailing spaces
- Remove additional blank lines
- Fix indentation consistency and correctness

* Update addons-examples.md

Co-authored-by: Maximilian Hils <github@maximilianhils.com>
 2021-03-11 19:13:02 +01:00
Maximilian Hils
70223163de
Merge pull request #4486 from mhils/websocket 
Merge WebSocketFlow into HTTPFlow, add WebSocket UI
 2021-03-11 11:02:40 +01:00
Maximilian Hils
d8aeef1bfd ressurect killed, is_text -> type 2021-03-10 20:29:16 +01:00
Maximilian Hils
e1f938f05b tests++ 2021-03-09 00:55:47 +01:00
Maximilian Hils
eb65bc748b merge WebSocketFlow into HTTPFlow, add WebSocket UI 2021-03-08 19:36:18 +01:00
Maximilian Hils
6d9f00408c
remove old socks proxy code (#4485) 2021-03-08 16:21:40 +00:00
Maximilian Hils
e2772f3eec fixup coverage 2021-03-07 22:35:53 +01:00
Maximilian Hils
d6cd1489d6 [sans-io] add SOCKS5 proxy mode 2021-03-07 22:19:03 +01:00
Maximilian Hils
93de96a720
don't set IP addresses as SNI (#4480) 2021-03-07 15:00:33 +00:00
Maximilian Hils
4bd7b6c4ea
speculative HTTP/2 fixes, refs #4451 (#4464) 2021-02-21 15:36:04 +01:00
Maximilian Hils
593dd93cf8
HTTP/2: Show human-readable error messages (#4462) 2021-02-20 12:49:21 +01:00
Maximilian Hils
001cf6c10a
memleak: immediately store errored flows, refs #4456 (#4461) 2021-02-19 21:23:42 +00:00
Maximilian Hils
cfaba15c1f
fix regression introduced in 70e08c880 (#4459) 2021-02-19 18:34:52 +00:00
Maximilian Hils
70e08c880c
don't reuse closed connections, refs #4451 (#4458) 2021-02-18 22:07:18 +00:00
Daniel Baskal
aebc40c408
Fixes #4416 (#4446) 
* Fixes #4416

Fix ValueError when splitting on a request URI without a path part.

* Fix mypy lintining issue

* Replace .split() with .partition() for cleaner code
 2021-02-14 18:19:00 +01:00
Maximilian Hils
748fc93699
Merge pull request #4422 from mhils/pdoc 
Docs: Add API Reference Using Pdoc
 2021-02-13 00:18:49 +01:00
Maximilian Hils
805aed4f6a docs++ 2021-02-13 00:13:37 +01:00
Denis Kasak
856a35af6d
Use original flow host instead of IP when exporting to curl/httpie. (#4307) 
Use original flow host instead of IP when exporting to curl/httpie.

Unless this is done, the SNI server name will not be sent, often making
the curl/httpie command have different behaviour than the original
request (most often in the form of failing to establish a TLS
connection).

With this change, we always use the original host, fixing this failure.
However, if the original host is a domain, it may sometimes resolve to
a different IP address later on. In curl, we solve this problem by
forcing it to connect to the original IP using `--resolve`. For httpie
there is currently no easy solution (see:
https://github.com/httpie/httpie/issues/414).
 2021-02-09 18:44:46 +00:00
Alexander Prinzhorn
4212a56f25
Collect all flow filter matches before modifying headers, fixes #4245 (#4246) 2021-02-09 19:37:46 +01:00
Maximilian Hils
0ab59e5524 docs++ 2021-02-05 22:04:45 +01:00
Maximilian Hils
ca578181e3 adjust tests, increase coverage 2021-02-05 08:44:17 +01:00
Maximilian Hils
dc6c5f55cd move Connections to mitmproxy.connection 2021-02-05 08:44:17 +01:00
Maximilian Hils
d68c364b35 inline/move http.make_* functions 2021-02-05 08:43:15 +01:00
Maximilian Hils
9409bf0368 move Headers/Message/Request/Response into mitmproxy.http 2021-02-05 08:43:15 +01:00
Maximilian Hils
50ae75aedf
Merge pull request #4413 from r00t-/compat_convert_fix 
fix conversion of old flows failing due to missing array keys
 2021-02-03 19:54:24 +01:00
Maximilian Hils
bd0e5989e2 add websocket flow conversion test 2021-02-03 19:27:15 +01:00
Maximilian Hils
faeb9678f1 test dumpfiles: .bin -> .mitm 2021-02-03 19:27:15 +01:00
Maximilian Hils
fccc153fdb
api docs++ (#4421) 2021-02-03 00:14:54 +01:00
Graham Robbins
2694b05fe9
Extract port number from authority before IDNA decode (#4410) 
* Extract port from authority before IDNA decode

A UnicodeError exception may be raised if the port is present

* Update Changelog

* Test for badly formed byte input
 2021-01-24 20:45:01 +01:00
Thomas B Homburg
4be69a5fc7
GraphQL content-view (#4396) 
* Add GraphQL contentview

* Simpler graphql formatting

* Do not prompt for exit

* Give graphql priority over json

* Revert "Do not prompt for exit"

This reverts commit 386b38f9eb4601ae41365ace5b9689c30761a412.

* Handle batched queries

* Fix flake issues

* typing fix

* Typing fixes

* Add graphql content view test

* Fix reference

* Fixed tests

* flake

* Implement GraphQLView.render_priority

* Show query key stub value

* Code coverage

* Import PARSE_ERROR

* Test for invalid json
 2021-01-22 00:29:09 +01:00
Kamil Borzym
f51019cb63
[#4235] Automatic view mode based on should_render method (#4236) 
* [#4235] Automatic view based on should_render method instead of content_types property

* [#4235] Update CHENGELOG

* [#4235] Fix linter warnings

* Add an explicit test for the new forward-compatible behaviour

* wip

* contentviews: introduce render_priority (2/2)

* coverage++, lint!

* minor fixes

Co-authored-by: Maximilian Hils <git@maximilianhils.com>
 2021-01-17 11:39:51 +01:00
Maximilian Hils
8dda8fc0f7 duplicate hooks: warn instead of raise 
raising causes issues when using pdoc's live reloading, this should fix it.
 2021-01-16 20:31:50 +01:00
Maximilian Hils
ff624d0b87 fix #4387 2021-01-06 10:47:37 +01:00
Maximilian Hils
81f8f06a11 converge on hooks, events, and commands 2021-01-05 22:00:54 +01:00
Maximilian Hils
6c0e4f1cb7 streamline event/hook/command naming conventions 2021-01-05 21:54:44 +01:00
Maximilian Hils
90df4168f8 coverage++ 2021-01-05 21:54:44 +01:00
Maximilian Hils
f9d18745c9 add MitmproxyEvent for addon events 
We previously defined the list of all allowed events in mitmproxy.eventsequence,
without specifying the expected arguments at all. This commit changes our addon
event dispatch to subclasses of MitmproxyEvent, which have well-defined attributes.
 2021-01-05 21:54:44 +01:00
Maximilian Hils
9186c1aeb5 add Client/Server.__str__ 2021-01-05 21:54:44 +01:00
Maximilian Hils
cca242a581 Windows: build pyinstaller onedir and use it for installer 
This greatly improves startup time as pyinstaller doesn't have to
unpack everything on startup. The same also applies to macOS and Linux,
but there we 1) don't have installers to hide all the files and
2) have a filesystem that deals much better with lots of small files.

Additionally, simplify cibuild to be a bit more reasonable.
 2021-01-05 15:56:33 +01:00
Maximilian Hils
ca45548289
Merge pull request #4384 from mhils/mitmproxy.exe 
mitmproxy.exe 🎉🥳
 2021-01-05 08:18:11 +01:00
Maximilian Hils
6cc452db8c
Merge pull request #4381 from mhils/options-cleanup 
Remove unused options, warn about nonfunctioning ones.
 2021-01-05 07:52:14 +01:00
Maximilian Hils
605961c57f build mitmproxy.exe 🎉 2021-01-05 07:46:53 +01:00
Maximilian Hils
63b8685356 add console integration tests 2021-01-04 20:21:42 +01:00
Maximilian Hils
455fee1126 warn about nonfunctioning options, permanently remove unused ones. 
If you are affected by this change, please do reach out by filing an issue.
 2021-01-01 18:34:15 +01:00
Maximilian Hils
67a93239f4 tlsconfig: respect add_upstream_certs_to_client_chain 2021-01-01 18:31:50 +01:00
Maximilian Hils
d8c3e8a56a
Merge pull request #4379 from mhils/reverse-tcp 
Reverse proxy: do not assume TLS for non-standard ports
 2021-01-01 18:08:38 +01:00
Maximilian Hils
2b2b40d53e [sans-io] respect websocket option 2021-01-01 17:35:34 +01:00
Maximilian Hils
0bbb5ca345 reverse proxy: do not assume TLS for non-standard ports 2021-01-01 16:57:50 +01:00
Maximilian Hils
dfba6e81a6 alpn: str -> bytes 2020-12-30 22:52:07 +01:00
Maximilian Hils
8ac5af62f5 cert test coverage++ 2020-12-30 22:29:56 +01:00
Maximilian Hils
abbe9eeb79 sni/alpn: bytes -> str 2020-12-30 20:26:31 +01:00
Maximilian Hils
2a30172525 migrate certs.Cert to pyca/cryptography, re-add mitmcert 2020-12-30 15:20:49 +01:00
Maximilian Hils
48b166ab57 move cert serialization to cryptography 2020-12-29 23:36:12 +01:00
Maximilian Hils
3fe29b27be use cryptography to generate certificates 
this fixes #4368, but we are not done here. the goal is to replace most
usages of pyOpenSSL's cert object with cryptography.
 2020-12-29 01:19:58 +01:00
Maximilian Hils
74aeb3ae4c add memory leak detect script for certificate generation, refs #4368 2020-12-29 01:17:01 +01:00
Maximilian Hils
70f1d173e2 Reply: remove return value 2020-12-28 22:09:57 +01:00
Maximilian Hils
1655f54817 cleanup old proxy server 2020-12-28 14:33:10 +01:00
Maximilian Hils
a0f923aa14 coverage++ 2020-12-28 09:56:25 +01:00
Maximilian Hils
2db9a43fd6 add type annotations, test sslkeylogfile 2020-12-28 09:44:37 +01:00
Maximilian Hils
de46db53e9 minor fixes 2020-12-21 20:52:02 +01:00
Maximilian Hils
57f5d1af63 monkeypatch pyOpenSSL to accept new TLS methods 2020-12-21 16:34:37 +01:00
Maximilian Hils
8842abbb4f clean up tls bits 2020-12-21 16:20:03 +01:00
Maximilian Hils
61a2beac3c minor fixes 2020-12-20 14:17:43 +01:00
Maximilian Hils
8ce4bc7142 additional test speedups 2020-12-20 00:50:40 +01:00
Maximilian Hils
2f725e5f02 massively speed up tests by properly mocking ldap proxyauth 2020-12-20 00:34:30 +01:00
Maximilian Hils
b05c13daa6 remove old mitmproxy.net.tcp code 
this is not needed anymore with sans-io
 2020-12-20 00:12:21 +01:00
Maximilian Hils
1ffe406d24
Merge pull request #4359 from mhils/warn-no-nextlayer 
Warn if no nextlayer addon is present
 2020-12-18 03:08:21 +01:00
Thomas Kriechbaumer
47f8e1d115 asgi: refactor tests to new proxy core 2020-12-15 23:37:39 +01:00
Maximilian Hils
e0d5dea959 warn if no nextlayer addon is present 2020-12-15 23:35:43 +01:00
Thomas Kriechbaumer
df164f42e1 imports cleanup 2020-12-14 23:52:48 +01:00
Thomas Kriechbaumer
722d8f2db7 onboardingapp: test using nativ flask test client 2020-12-14 23:52:48 +01:00
Thomas Kriechbaumer
60342344e4 nuke sessions addon with sqlite an protobuf components 2020-12-14 22:14:44 +01:00
Thomas Kriechbaumer
acd985c4c9 fix extracting PEM from context.Server.certificate_list 2020-12-14 22:14:44 +01:00
Thomas Kriechbaumer
55a58ba8e8 fix deprecation warnings in context client/server 2020-12-14 22:14:44 +01:00
Thomas Kriechbaumer
d159897d98 nuke legacy protocol stack 2020-12-14 22:14:43 +01:00
Thomas Kriechbaumer
e6445af2cd cibuild: cleanup outdated providers 2020-12-14 20:01:32 +01:00
Thomas Kriechbaumer
c35316f85a remove pathod and pathoc 2020-12-13 20:21:11 +01:00
Maximilian Hils
a3909ee024 [sans-io] add dedicated proxy_debug option 2020-12-12 23:57:20 +01:00
Maximilian Hils
a768825fba [sans-io] docs++ 2020-12-12 21:55:58 +01:00
Maximilian Hils
8f49cde627 [sans-io] add tunnel tests 2020-12-12 21:24:21 +01:00
Maximilian Hils
f0bdf887fc [sans-io] tests++ 2020-12-12 21:24:21 +01:00
Maximilian Hils
641b1c425b [sans-io] add mode tests, minor fixes 2020-12-12 21:24:21 +01:00
Maximilian Hils
4c75765387 [sans-io] test coverage++ 2020-12-12 21:24:21 +01:00
Maximilian Hils
64d2ac8ef3 [sans-io] add Layer tests 2020-12-12 21:24:21 +01:00
Maximilian Hils
ed753d1a38 [sans-io] add read_sansio tests 2020-12-12 21:24:21 +01:00
Maximilian Hils
d3bd603fb9 [sans-io] add proxyserver tests 2020-12-12 21:24:21 +01:00
Maximilian Hils
8e7cbb3991 [sans-io] remove unused code 2020-12-12 21:24:21 +01:00
Maximilian Hils
efacbca0ca [sans-io] add clientplayback tests 2020-12-12 21:24:21 +01:00
Maximilian Hils
1c9e9660b8 [sans-io] add next_layer tests 2020-12-12 21:24:21 +01:00
Maximilian Hils
21293edce0 [sans-io] TLS: add handshake fuzzing 2020-12-12 21:24:21 +01:00
Maximilian Hils
de925ab397 [sans-io] fix tests 2020-12-12 21:24:21 +01:00
Maximilian Hils
8201a90e22 [sans-io] lint! 2020-12-12 21:24:21 +01:00
Maximilian Hils
00c54e68b8 [sans-io] HTTP/2: more fuzzing, improve cancellation logic 2020-12-12 21:24:21 +01:00
Maximilian Hils
8f516bfd81 [sans-io] HTTP/2: reset half-closed streams on error 2020-12-12 21:24:21 +01:00
Maximilian Hils
66868b1186 [sans-io] HTTP/2: protocol fixes, tests++ 2020-12-12 21:24:21 +01:00
Maximilian Hils
52479e94bb [sans-io] include non-standard ports in host header, refs #4280 2020-12-12 21:24:21 +01:00
Maximilian Hils
8ae2ab2aca [sans-io] fix HTTP/2 stream cancellation 2020-12-12 21:24:20 +01:00
Maximilian Hils
d32a5d5f33 [sans-io] websockets: fix bugs, 100% test coverage 🎉 2020-12-12 21:24:20 +01:00
Maximilian Hils
a4a0428bc6 [sans-io] handle 101 switching protocols 2020-12-12 21:24:20 +01:00
Maximilian Hils
19f8cd0839 [sans-io] add websocket proof-of-concept 2020-12-12 21:24:20 +01:00
Maximilian Hils
222d5cedc8 [sans-io] HTTP/1: Add tests for HTTP -> TCP pipe behavior 2020-12-12 21:24:20 +01:00