Grasscutters/mitmproxy
3
0
Fork 0
mirror of https://github.com/Grasscutters/mitmproxy.git synced 2024-11-23 08:11:00 +00:00
Code Issues Projects Releases Wiki Activity
6,159 Commits 1 Branch 67 Tags 50 MiB
4158a1ae55
Commit Graph

1375 Commits

Author SHA1 Message Date
Aldo Cortesi
e387c68b38 console: mark view indiciator, fix empty flow list behaviour 
Fixes #1782
 2016-12-16 11:57:45 +13:00
Aldo Cortesi
6b5673e849 Merge pull request #1845 from mhils/mitmweb-improvements 
Mitmweb Improvements
 2016-12-16 10:04:12 +13:00
Maximilian Hils
727abdba44 Merge pull request #1852 from mhils/simplenamespace 
Use types.SimpleNamespace instead of custom NS class
 2016-12-13 15:06:14 +01:00
Maximilian Hils
44f94c8844 use types.SimpleNamespace instead of custom NS class 2016-12-13 14:23:21 +01:00
Maximilian Hils
e5b3c8bed3 fix #1850 2016-12-13 14:12:18 +01:00
Maximilian Hils
d854e08653 [web] various fixes 2016-12-12 00:08:29 +01:00
Aldo Cortesi
a99bf0814c console: clean path prompt handling 
- Don't trigger callback if no path is specified
- Expand userdir centrally
- Catch IO errors from callbacks and turn them into status messages
 2016-12-12 09:13:43 +13:00
Aldo Cortesi
ca33bea296 Extend type checker validate Sequence specs 2016-12-12 07:39:59 +13:00
Maximilian Hils
3e37cbd061 minor fixes 2016-12-10 12:06:33 +01:00
Maximilian Hils
123ef043dc add new xml/html pretty-printer 🎉 2016-12-10 11:40:07 +01:00
Maximilian Hils
293b79af91 remove lxml-dependent code 2016-12-10 10:19:05 +01:00
Maximilian Hils
a7ba2f7b46 update tests, increase coverage, add type info 2016-12-09 21:26:02 +01:00
Aldo Cortesi
e943147fc3 options: add a merge method to assimilate options 
Fixes #953
 2016-12-08 16:12:40 +13:00
Aldo Cortesi
b2695dbc6a Minor tweaks 2016-12-08 10:21:06 +13:00
Aldo Cortesi
a617e3b5f7 Amalgamate all the Options objects 2016-12-08 10:21:06 +13:00
Aldo Cortesi
d742d4fb8c Replace configargparse with argparse, enable new options loading 
This commit ditches configargparse and enables the new config file format. The
default location is ~/.mitmproxy/config.yaml. Unifying all of the various
Options objects will follow in the next patch.
 2016-12-08 10:21:06 +13:00
Aldo Cortesi
150372e297 options: defaults are a class attribute 2016-12-08 10:21:06 +13:00
Aldo Cortesi
00c897a185 options: save defaults, add .reset() to restore defaults 
Use .reset() in console app to clear options.
 2016-12-08 10:21:06 +13:00
Aldo Cortesi
d74cac265a Add YAML serialization of options 
This uses ruamel.yaml. The library seems well-supported, and can do in-place
modification of config files that retains user comments and file structure.
 2016-12-08 10:21:06 +13:00
Aldo Cortesi
c94cd512d1 options: defaults are a class attribute 2016-12-08 10:21:06 +13:00
Aldo Cortesi
c062e302e9 Add OptManager.subscribe, use it to clean up palettes in console 
.subscribe lets you subscribe a function to a specified set of options.
 2016-12-08 10:21:06 +13:00
Aldo Cortesi
0c091bd92b Options - avoid mutation, API cleanup, has_changed 
- Always return a deepcopy of options to avoid accidental mutation of options
state.
- Remove .get(opt, default). This is an inappropriate API for Options - trying
to retrieve an option that doesn't exist should always be an error.
- Add the has_changed method that checks if an option differs from the default,
use it in mitmproxy console.
 2016-12-08 10:21:06 +13:00
Aldo Cortesi
b231836c70 Get rid of tfile testing option 
It's weird, it's ugly, it's getting in the way of my options refactoring, and
it must therefore die.
 2016-12-08 10:21:05 +13:00
Aldo Cortesi
297493801d Clean up dump tests 
- Remove tests that redundantly test addon funtionality that we've already
tested more comprehensively elsewhere.
- Extend to 100% coverage for tools/dump.py
 2016-12-08 10:21:05 +13:00
Aldo Cortesi
b4d33aaebf options: save defaults, add .reset() to restore defaults 
Use .reset() in console app to clear options.
 2016-12-08 10:21:05 +13:00
Thomas Kriechbaumer
47196e8676 tls_parser: allow optional extensions 
fixes #1816
 2016-12-06 19:51:57 +01:00
Thomas Kriechbaumer
1a36efbb6a simplify ALPN and OpenSSL on macOS 2016-12-04 11:04:06 +01:00
Thomas Kriechbaumer
cc6aa1f542 websocket: update close handshake tests 2016-11-29 23:47:19 +01:00
Maximilian Hils
34bf3a2496 fix #1783 (#1794) 2016-11-28 13:29:06 +01:00
Maximilian Hils
3154dc87fd fix pyinstaller 
amongst other things, fix #1746
 2016-11-28 00:51:52 +01:00
Maximilian Hils
568f40c810 Fix transparent mode initialisation, refactor mitmproxy.platform (#1787) 2016-11-25 17:32:23 +01:00
Maximilian Hils
222106916e Merge branch 'mitmweb-eventlog' 2016-11-23 22:47:50 +01:00
Maximilian Hils
45332006a3 mitmweb: 100% app test coverage, numerous fixes 2016-11-23 22:35:07 +01:00
Thomas Kriechbaumer
d3bd04dec0 disable TCP support for now 2016-11-23 10:18:45 +01:00
Thomas Kriechbaumer
5dfc199086 websocket: add tests 2016-11-23 10:18:45 +01:00
Thomas Kriechbaumer
4beb693c9c websocket: change cmdline option 2016-11-23 10:18:45 +01:00
Thomas Kriechbaumer
ffb3988dc9 rename WebSocket{s,} protocol 2016-11-23 10:18:45 +01:00
Maximilian Hils
f1662cbfd7 add eventstore tests 2016-11-23 01:27:51 +01:00
Yoginski
ebff5f2466 Descriptive error message for SSL context initialization failure (#1767) 2016-11-21 09:40:09 +01:00
Maximilian Hils
9af8f4bb31 organize examples 
This commit is largely based on work by Thiago Arrais (@thiagoarrais)
and Shane Bradfield (@l33tLumberjack). I wasn't really able to get their
PR reasonably merged onto the latest master, so I reapplied their changes
manually here and did some further improvements on that.
 2016-11-21 02:28:10 +01:00
Maximilian Hils
f74e561524 rewrite absolute-form HTTP requests to relative form, fix #1759 (#1765) 2016-11-20 17:21:21 +01:00
Thomas Kriechbaumer
f45034e8f1 tests: fix race condition 2016-11-20 17:02:26 +01:00
Maximilian Hils
5ce370e2a9 lint! 2016-11-18 12:59:59 +01:00
Aldo Cortesi
8065b44eed addons.view: clarify modification events 
This adds a set of store modification events, and uses them for flow settings.
This addresses a bug where settings could persist even after flows were deleted.
 2016-11-15 09:53:17 +13:00
Aldo Cortesi
9b08279c7c addons.proxyauth: out with the old, in with the new 
- Strip out old auth mechanisms, and enable addon
- Disable web app auth for now - this should just use the Tornado auth stuff
 2016-11-13 19:59:59 +13:00
Aldo Cortesi
dc88b7d110 addons.proxyauth: complete and test 2016-11-13 18:14:23 +13:00
Aldo Cortesi
e644d2167c stub out proxyauth addon 
Stub out basic workings, add and test configure event.
 2016-11-13 16:22:33 +13:00
Aldo Cortesi
fe01b1435a upstream_proxy_auth -> upstream_auth 
Also clarify what this does in commandline help.
 2016-11-13 11:50:28 +13:00
Aldo Cortesi
3b00bc339d Complete upstream authentication module 
- Handles upstream CONNECT and regular requests, plus HTTP Basic for reverse
proxy
- Add some tests to make sure we can rely on the .via attribute on server
connections.
 2016-11-13 11:43:27 +13:00
Aldo Cortesi
a9b4560187 Refine handling of HTTP CONNECT 
- CONNECT requests do not generate the usual http events. Instead, they
generate the http_connect event and handlers then have the option of setting an
error response to abort the connect.
- The connect handler is called for both upstream proxy and regular proxy CONNECTs.
 2016-11-12 18:28:37 +13:00
Aldo Cortesi
38f8d9e541 Add the http_connect event for HTTP CONNECT requests 2016-11-12 12:44:43 +13:00
Aldo Cortesi
bc01a146b0 Upstream proxy auth to addon 2016-11-12 11:58:04 +13:00
Aldo Cortesi
3958940420 Test failure during 100-continue 
Also:

- Remove duplicate and unused code
- Tighten scope of HttpReadDisconnect handler - we only want to ignore this for
the initial read, not for the entire block that includes things like the expect
handling.
 2016-11-12 11:58:04 +13:00
Aldo Cortesi
82ac7d05a6 Bug: ask requestheaders before request body is read 
Also add the beginnings of a test suite to exercise issues like this.
 2016-11-12 11:58:04 +13:00
Thomas Kriechbaumer
184e29e119 http2: add push-promise metadata test 2016-11-05 05:13:36 +01:00
Aldo Cortesi
067198a5dd Remove deprecated interfaces 2016-11-04 11:47:47 +13:00
Aldo Cortesi
6c8c4465d9 script.concurrent: tests to taddons 2016-11-04 10:59:41 +13:00
Aldo Cortesi
4f4db223fe addons: convert last of the test suite to taddons 2016-11-04 09:17:56 +13:00
Aldo Cortesi
deb66d3cac addons.setheaders: fix configure bug, tests to taddons 2016-11-04 09:09:39 +13:00
Aldo Cortesi
b51a96081a addons.view: test suite to 100% 
This takes coverage for all of /addons to 100%
 2016-11-04 08:55:01 +13:00
Aldo Cortesi
17e828b243 addons.view: test improvements 2016-11-03 20:53:06 +13:00
Aldo Cortesi
5ade93f2ad addons.streambodies: 100% test coverage 2016-11-03 20:16:07 +13:00
Aldo Cortesi
d7d6edb3d1 addons.serverplayback: 100% test coverage 2016-11-03 20:05:21 +13:00
Aldo Cortesi
e1fc80937d addons.serverplayback: test suite to taddons 2016-11-03 19:43:33 +13:00
Aldo Cortesi
e9a96f4d7f addons.script: 100% test coverage 2016-11-03 19:36:34 +13:00
Aldo Cortesi
d9538637c3 addons.script: convert to test.taddons 2016-11-03 18:57:34 +13:00
Aldo Cortesi
d31f2698a5 addons.onboarding: remove duplicate code 2016-11-03 16:20:38 +13:00
Aldo Cortesi
9fc6674151 addons.intercept: 100% test coverage 2016-11-03 16:16:29 +13:00
Aldo Cortesi
77e6dfe35c streamfile: make options less crazy 
Instead of a (file, mode) tuple, move to "streamfile" as the path, and
"streamfile_append" to specify if we should append or not.

Leave the command-line options intact for now.
 2016-11-03 15:18:55 +13:00
Aldo Cortesi
9f77c79227 FileStreamer -> StreamFile 
options.wfile -> options.streamfile
 2016-11-03 15:00:18 +13:00
Aldo Cortesi
43a8221989 addons: filestreamer test cov to 100% 
Also address a bug in taddons that caused options changes that raised errors to
be retained rather than rolled back.
 2016-11-03 14:50:23 +13:00
Aldo Cortesi
fbdce4b629 addons filstreamer: tests and bugfixes 
- Move tests to taddons
- Fix a bug where options updates caused streaming to halt
 2016-11-03 12:44:19 +13:00
Aldo Cortesi
5109fd8ecb Change antiquated test cipher suites 
This breaks Travis, possibly due to OpenSSL version.
 2016-11-03 08:23:19 +13:00
Aldo Cortesi
b867fb35a3 addons: dumper spit and polish 
- 100% test coverage
- Cleanups
- Add test/mitmproxy/addons/dumperview.py, a small utility for viewing dumper
output variations
 2016-11-02 22:47:49 +13:00
Aldo Cortesi
a75b3474a4 tests: convert test_dumper to use taddons 2016-11-02 11:17:08 +13:00
Aldo Cortesi
59b269425f tests: client_playback coverage 2016-11-02 10:59:33 +13:00
Aldo Cortesi
7e6d014f8f tests: pathod/tutils.py -> pathod/tservers.py 
And remove all aliases for mitmproxy.test.tutils
 2016-11-02 10:06:25 +13:00
Aldo Cortesi
c55e8d8f62 tests: clean up test/mitmproxy/tutils.py 
Remove aliases, use mitmproxy/test/tutils instead.
 2016-11-02 09:44:18 +13:00
Aldo Cortesi
fc7606bd98 tests: minor cleanup for tests/mitmproxy/tutils.py 2016-11-02 08:51:26 +13:00
Aldo Cortesi
83fdd82a52 addons anticom, anticache: expand tests, new truss 2016-11-01 17:38:45 +13:00
Aldo Cortesi
50deaf56bf Remove addons.State 
- Do a first-pass port of mitmweb. Some stuff still broken.
- Remove more all State tests from the test suite
 2016-11-01 10:06:28 +13:00
Aldo Cortesi
9d9735dd07 Replace addons.state in test suite 
Use a super-simple flow recorder instead.
 2016-11-01 09:36:24 +13:00
Aldo Cortesi
daf355bb4c console: add caching sort keys 
This is a tad complicated. The underlying sorted list implementation expects
the sort key to be stable for the lifetime of the object. However, if we sort
by size, for instance, the sort order changes as the flow progresses through
its lifecycle. We address this through two means:

- Let order keys cache the sort value by flow ID.
- Add a facility to refresh items in the list by removing and re-adding them
when they are updated.
 2016-10-30 21:11:30 +13:00
Aldo Cortesi
9abfb1aac2 console: re-add follow focus 2016-10-30 19:47:32 +13:00
Aldo Cortesi
f08b57fb9b console: "v" to reverse flow order 2016-10-30 17:57:43 +13:00
Aldo Cortesi
a67a591893 console: o for flow order 
"o" shortcut and --order cmdline option to change flow order.

Options is now bound to "O".
 2016-10-30 17:42:34 +13:00
Aldo Cortesi
6b6c44551a addons.view: Add first-class support for marking 
Marking now works differently - it's no longer just another filter, it's
applied in addition to the filter. This means you can apply a filter, mark some
flows, and then toggle between marked and unmarked flows matching the filter.

I'm leaving the filter for marked flows intact - it will come in handy in other
situations.
 2016-10-30 15:45:36 +13:00
Aldo Cortesi
2b76db1272 console: Port from state to view 
First phase of the port - basic flow list and flow view functionality working.
More to come.
 2016-10-30 14:56:36 +13:00
Aldo Cortesi
b9eb1a3479 Add options.default_contentview 
This is the first step in a longer process of revamping content views. For the
moment, the option is not exposed on the command line.
 2016-10-30 12:05:37 +13:00
Aldo Cortesi
944dcbaaa0 Cleanups: remove intercept from addons.state, unused imports 2016-10-30 11:13:05 +13:00
Aldo Cortesi
81a00f6f76 Replace interception in mitmproxy and mitmweb with addon 
Fixes #1674
 2016-10-30 11:07:09 +13:00
Aldo Cortesi
d1f14961ed test.taddon: Add cycle() method, use it to test addons.intercept 2016-10-30 10:52:57 +13:00
Aldo Cortesi
b4904d33ba Move test flow creators to mitmproxy.test.tflow 2016-10-30 10:12:39 +13:00
Aldo Cortesi
2dc3284fbb Add addons.intercept 
- Add an addon to handle intercept based on a filter pattern
- Start sketching out a nicer testing truss for addons in mitmproxy.test.taddon
 2016-10-29 15:44:48 +13:00
Aldo Cortesi
71d2636594 Sync 2016-10-29 12:35:10 +13:00
Aldo Cortesi
005c22445b addons.view: focus.focusflow -> focus.flow 2016-10-29 11:55:44 +13:00
Aldo Cortesi
7ecaeb0214 addons.view.focus: next and prev methods 2016-10-29 11:50:10 +13:00
Aldo Cortesi
32a0a7b860 addons.view: flow settings 
Add a flow settings mechanism, enable focus and settings unilaterally.
 2016-10-29 11:37:32 +13:00
Aldo Cortesi
14df969434 addons.view.focus: Better handling of view refresh 
When we refresh and our current focus goes out of scope, we set the focus to
the element nearest the old focus.
 2016-10-29 11:08:35 +13:00
Aldo Cortesi
90e7142b5c addons.View: better order reversal 
Deal with some subtleties in order reversal, add a toggle method that emits
refresh.
 2016-10-29 10:55:57 +13:00
Aldo Cortesi
12a70d03ad addons.view: Add a focus tracker 2016-10-29 10:17:29 +13:00
Aldo Cortesi
9dcc3a3e20 addons.View: hook up signals 2016-10-29 08:25:19 +13:00
Aldo Cortesi
69bacee1d8 Sketch out addons.View 
The first iteration of a replacement for addons.State
 2016-10-29 08:25:19 +13:00
Thomas Kriechbaumer
44ac370f08 upgrade h2 to >=2.5.0 2016-10-27 15:10:53 -07:00
Maximilian Hils
97f1236c99 Merge pull request #1670 from chhsiao90/display-cipher-#582 
Resolved #582: display ClientConnection select cipher of TLS
 2016-10-27 11:38:22 -07:00
chhsiao90
d52f35428c Show tls version on console 2016-10-26 14:36:14 +08:00
Maximilian Hils
b1bdae3d1c typecheck options, fix current inconsistencies 2016-10-25 20:45:48 -07:00
chhsiao90
960f2e8bf0 Resolved #582: display ClientConnection select cipher of TLS 2016-10-26 11:32:42 +08:00
Maximilian Hils
145c2892f7 Merge pull request #1664 from chhsiao90/sni-display-#1639 
Resolved #1639: display sni on ClientConnection
 2016-10-24 19:56:05 -07:00
chhsiao90
39ac29e37c Resolved #1639: display sni on ClientConnection 2016-10-25 10:46:53 +08:00
Maximilian Hils
ef4e9b2b85 Merge pull request #1656 from mhils/improve-export-2 
Improve Flow Export
 2016-10-24 19:19:58 -07:00
Sahil Chelaramani
7b3505336a replace mitmproxy.utils.lrucache with functools.lru_cache 2016-10-24 16:03:16 -07:00
Maximilian Hils
e87daa70f3 improve flow export 2016-10-22 20:32:39 -07:00
Maximilian Hils
ea2d6474bf Merge pull request #1649 from cortesi/kill 
Don't call error for flow kill
 2016-10-22 19:22:50 -07:00
Maximilian Hils
a1a792aeac various encoding fixes, fix #1650 2016-10-22 18:47:12 -07:00
Aldo Cortesi
6170493615 Don't call error for flow kill 
This is now the error handler on master, so whatever the intetion was it's now
definitely wrong.
 2016-10-22 15:56:42 +13:00
Aldo Cortesi
cc8b422d9d multidict: remove to_dict 
We never use it, and it is dangerously ambiguous when a key is associated with
a list.
 2016-10-21 11:42:21 +13:00
Aldo Cortesi
18ee6255c0 multidict: ditch ImmutableMultiDict 
A contorted class we only use for cookie attributes. We don't need it.
 2016-10-21 11:40:05 +13:00
Aldo Cortesi
8430f857b5 The final piece: netlib -> mitproxy.net 2016-10-20 11:56:38 +13:00
Aldo Cortesi
853e03a5e7 netlib.tutils -> mitmproxy.test.tutils 
There's a LOT more to be done refactoring our different conflicting test utils.
 2016-10-20 11:42:55 +13:00
Aldo Cortesi
01a449b5cb netlib.exceptions.* -> mitmproxy.exceptions 2016-10-20 11:27:05 +13:00
Aldo Cortesi
301d52d9d0 netlib.encoding -> netlib.http.encoding 
Encoding is highly specific to http, and only used within this module.
 2016-10-20 11:06:57 +13:00
Aldo Cortesi
f964d49853 netlib.certutils -> mitmproxy.certs 2016-10-20 11:02:52 +13:00
Aldo Cortesi
9870844b38 netlib.utils -> netlib.check 
Now only contains is_valid_[host,port]

Intermediate step - this will be in mitproxy.net soon.
 2016-10-20 10:46:47 +13:00
Aldo Cortesi
e0f3cce14c netlib.utils.[get,set]bit -> mitmproxy.utils.bits 2016-10-20 10:38:57 +13:00
Aldo Cortesi
9555126585 netlib.utils.BiDi -> mitmproxy.types.bidi.BiDi 2016-10-20 10:32:09 +13:00
Aldo Cortesi
a684585e7c netlib.debug -> mitmproxy.utils.debug 2016-10-20 10:25:36 +13:00
Aldo Cortesi
1ecb25cdc1 mitmproxy.types.[basethread,multidict,serializable] 2016-10-20 10:22:23 +13:00
Aldo Cortesi
f45f4e677e netlib.strutils -> mitmproxy.utils.strutils 2016-10-20 10:11:58 +13:00
Aldo Cortesi
1407830280 netlib.human -> mitmproxy.utils.human 2016-10-20 09:45:18 +13:00
Aldo Cortesi
069119364d Create mitmproxy.utils hierarchy 
- Add mitproxy.utils.lrucache, mitproxy.utils.data
 2016-10-20 09:35:55 +13:00
Aldo Cortesi
e73c7fe77e mitmproxy.protocol -> mitmproxy.proxy.protocol 
The protocols here are compltely proxy-specific, are only used from within the
proxy module, and are not exposed to users.
 2016-10-19 23:11:56 +13:00
Aldo Cortesi
24cf8da27e Move all tools into mitmproxy.tools, move models/* to top level 
The primary motivation here (and for all the other moving around) is to present
a clean "front of house" to library users, and to migrate primary objects to
the top of the module hierarchy.
 2016-10-19 20:26:05 +13:00
Aldo Cortesi
5a68d21e8c Remove flow module entirely, move contents to top level 
mitmproxy.flow.io -> mitmproxy.io
mitmproxy.flow.export -> mitmproxy.export
 2016-10-19 15:08:35 +13:00
Aldo Cortesi
7c32d4ea2a flow.state -> addons.state 2016-10-19 14:48:42 +13:00
Aldo Cortesi
22eebfd574 addons.Addons -> addonmanager, builtins -> addons 2016-10-19 14:39:39 +13:00
Aldo Cortesi
966418725b controller.Log -> log.Log 2016-10-19 14:14:59 +13:00
Aldo Cortesi
bce387a5a0 Kill flow.master - create master.Master 
Also extract events into .events
 2016-10-19 13:22:50 +13:00
Aldo Cortesi
264a09e778 Let's dispense with this fiction that we support multiple servers 
It's simple enough to re-add if needed, and until then is just cruft.
 2016-10-19 12:37:32 +13:00
Aldo Cortesi
87629586ae web app cleanups: tests and examples 2016-10-19 11:48:51 +13:00
Aldo Cortesi
ceb8caee98 Web apps to addons 
This commit removes the app registry, adds a wsgiapp addon base, and ports the
onboarding app to it.
 2016-10-19 11:40:51 +13:00
Aldo Cortesi
03cb5bb325 No need for all builtins to be loaded for server tests 2016-10-18 22:30:21 +13:00
Aldo Cortesi
466f5e56a1 Convert flow.state to an addon 
A first step, and already many things are much nicer.
 2016-10-18 21:51:36 +13:00
Aldo Cortesi
d5056c5627 accept_addon -> resume, intercept/resume events 2016-10-18 12:02:35 +13:00
Aldo Cortesi
825b02d495 master.duplicate_flow -> state.duplicate_flow 
This is a pure operation on state, and doesn't belong on master.

Part of prep to move state to an addon.
 2016-10-18 11:29:51 +13:00
Aldo Cortesi
b25d94a6ac Response body streaming to addon 2016-10-18 10:42:05 +13:00
Maximilian Hils
beed406058 fix tests 2016-10-17 03:37:42 -07:00
Thomas Kriechbaumer
94c9dd66e6 remove empty lines at beginning of file 2016-10-17 10:32:12 +02:00
Maximilian Hils
758860531a fix socks5 mode with tls 2016-10-16 23:05:51 -07:00
Aldo Cortesi
c774a9fec9 python3: clean up super and __future__ 2016-10-17 18:03:07 +13:00