Commit Graph

1089 Commits

Author SHA1 Message Date
Thomas Kriechbaumer
1fcf79fff0 fix ALPN usage 2016-12-02 11:58:20 +01:00
Thomas Kriechbaumer
d2216801dd websocket: make flowfilter work 2016-11-29 23:01:55 +01:00
Thomas Kriechbaumer
ea6de424a3 websocket: carry over per-message compression bit 2016-11-29 22:32:55 +01:00
Thomas Kriechbaumer
073a286098 websocket: reduce connection timeout 2016-11-29 22:17:33 +01:00
Thomas Kriechbaumer
bd8ae910d2 websocket: fix message type on chunking 2016-11-29 22:13:59 +01:00
Thomas Kriechbaumer
6e15e766c5 websocket: fix close handshake and re-chunking 2016-11-29 22:01:41 +01:00
Thomas Kriechbaumer
5ee192b758 websocket: fix empty frame with fin=1 2016-11-29 21:00:42 +01:00
Maximilian Hils
34bf3a2496 fix #1783 (#1794) 2016-11-28 13:29:06 +01:00
Maximilian Hils
e64d2ce829 Merge pull request #1792 from Kriechi/mitmdump-filter-help
improve mitmdump filter help
2016-11-28 02:13:39 +01:00
Maximilian Hils
3154dc87fd fix pyinstaller
amongst other things, fix #1746
2016-11-28 00:51:52 +01:00
Thomas Kriechbaumer
b5daafb518 improve mitmdump filter help 2016-11-25 21:59:13 +01:00
Maximilian Hils
568f40c810 Fix transparent mode initialisation, refactor mitmproxy.platform (#1787) 2016-11-25 17:32:23 +01:00
Maximilian Hils
0386740404 mitmweb: include console logging 2016-11-25 17:20:14 +01:00
Maximilian Hils
1d3cb9eeb8 mitmweb: add --no-browser 2016-11-25 17:20:14 +01:00
Maximilian Hils
222106916e Merge branch 'mitmweb-eventlog' 2016-11-23 22:47:50 +01:00
Maximilian Hils
d15ddfad14 fix contentview attributes 2016-11-23 22:47:36 +01:00
Maximilian Hils
45332006a3 mitmweb: 100% app test coverage, numerous fixes 2016-11-23 22:35:07 +01:00
Thomas Kriechbaumer
9bc5adfb03 add more websocket indicators to mitmproxy and mitmdump 2016-11-23 10:18:45 +01:00
Thomas Kriechbaumer
4b04566a34 add metadata info to flow detail view 2016-11-23 10:18:45 +01:00
Thomas Kriechbaumer
ea97f62975 websocket: show messages in the eventlog 2016-11-23 10:18:45 +01:00
Thomas Kriechbaumer
5dfc199086 websocket: add tests 2016-11-23 10:18:45 +01:00
Thomas Kriechbaumer
4beb693c9c websocket: change cmdline option 2016-11-23 10:18:45 +01:00
Thomas Kriechbaumer
3d8f3d4c23 add WebSocket flows and messages 2016-11-23 10:18:45 +01:00
Thomas Kriechbaumer
ffb3988dc9 rename WebSocket{s,} protocol 2016-11-23 10:18:45 +01:00
Maximilian Hils
dc75605e46 minor fixes 2016-11-23 01:27:52 +01:00
Maximilian Hils
72dcf70db2 mitmweb: fix event display 2016-11-23 01:27:51 +01:00
Maximilian Hils
fea6041cde mitmweb: cleanup, reimplement eventlog 2016-11-23 01:27:50 +01:00
Maximilian Hils
a6c608e085 make the new mypy release happy 2016-11-23 01:27:23 +01:00
Maximilian Hils
40f0193dda remove mitmweb auth
9b08279c7c removed the actual functionality.
we should not have a command line switch that does nothing. :)
2016-11-22 18:27:16 +01:00
Maximilian Hils
21a03d56b5 don't set an empty ALPN, refs #1772 2016-11-22 17:02:37 +01:00
Maximilian Hils
f644665cd9 update pydivert, fix #1749 2016-11-21 14:14:52 +01:00
Yoginski
ebff5f2466 Descriptive error message for SSL context initialization failure (#1767) 2016-11-21 09:40:09 +01:00
Maximilian Hils
f74e561524 rewrite absolute-form HTTP requests to relative form, fix #1759 (#1765) 2016-11-20 17:21:21 +01:00
Edgar Boda-Majer
078f36d86a handle SIGINT with a quit prompt (#1760) 2016-11-20 16:40:04 +01:00
Doug Freed
0ff1967226 proxy/protocol/base: Allow spoofing fixed source
Allow combining --spoof-source-address and --upstream-bind-address to
spoof a fixed source address.  Also update the help text for
--spoof-source-address to indicate this new possibility.
2016-11-18 14:46:32 +00:00
Maximilian Hils
5ce370e2a9 lint! 2016-11-18 12:59:59 +01:00
Maximilian Hils
2ee8bc2f1a fix type checks 2016-11-16 17:19:38 +01:00
Maximilian Hils
de6bf175e2 mitmweb: start browser on startup 2016-11-16 15:09:54 +01:00
Maximilian Hils
677789a617 minor fixes 2016-11-16 15:09:53 +01:00
Maximilian Hils
47ec1c9570 Merge pull request #1742 from dwfreed/patch-2
Make the upstream bind address a separate option
2016-11-16 13:37:12 +01:00
Thomas Kriechbaumer
3d26bd4aa1 Merge pull request #1748 from Kriechi/better-onboarding
add macOS/OSX installation instructions to onboarding page
2016-11-16 13:04:07 +01:00
Thomas Kriechbaumer
d7a22d92ec add macOS/OSX installation instructions to onboarding page 2016-11-16 11:05:52 +01:00
Aldo Cortesi
8065b44eed addons.view: clarify modification events
This adds a set of store modification events, and uses them for flow settings.
This addresses a bug where settings could persist even after flows were deleted.
2016-11-15 09:53:17 +13:00
Doug Freed
4cfda51c37 proxy/protocol/base: use upstream_bind_address
Use the upstream_bind_address option, instead of listen_host.
2016-11-14 12:12:48 +00:00
Doug Freed
f89671a33b tools/cmdline: add upstream bind address option
Allow specifying the upstream bind address on the command line.
2016-11-14 12:12:48 +00:00
Doug Freed
078bd532c3 options: add upstream_bind_address
Add upstream_bind_address to the options object
2016-11-14 12:12:48 +00:00
Aldo Cortesi
2d0a65a3f4 Merge pull request #1738 from Kriechi/fix-1737
restrict ALPN offers if host changes
2016-11-14 08:50:29 +13:00
Thomas Kriechbaumer
6fb706ec15 restrict ALPN offers if host changes 2016-11-13 12:11:56 +01:00
Aldo Cortesi
9b08279c7c addons.proxyauth: out with the old, in with the new
- Strip out old auth mechanisms, and enable addon
- Disable web app auth for now - this should just use the Tornado auth stuff
2016-11-13 19:59:59 +13:00
Aldo Cortesi
dc88b7d110 addons.proxyauth: complete and test 2016-11-13 18:14:23 +13:00
Aldo Cortesi
e644d2167c stub out proxyauth addon
Stub out basic workings, add and test configure event.
2016-11-13 16:22:33 +13:00
Aldo Cortesi
fe01b1435a upstream_proxy_auth -> upstream_auth
Also clarify what this does in commandline help.
2016-11-13 11:50:28 +13:00
Aldo Cortesi
3b00bc339d Complete upstream authentication module
- Handles upstream CONNECT and regular requests, plus HTTP Basic for reverse
proxy
- Add some tests to make sure we can rely on the .via attribute on server
connections.
2016-11-13 11:43:27 +13:00
Aldo Cortesi
a9b4560187 Refine handling of HTTP CONNECT
- CONNECT requests do not generate the usual http events. Instead, they
generate the http_connect event and handlers then have the option of setting an
error response to abort the connect.
- The connect handler is called for both upstream proxy and regular proxy CONNECTs.
2016-11-12 18:28:37 +13:00
Aldo Cortesi
38f8d9e541 Add the http_connect event for HTTP CONNECT requests 2016-11-12 12:44:43 +13:00
Aldo Cortesi
bc01a146b0 Upstream proxy auth to addon 2016-11-12 11:58:04 +13:00
Aldo Cortesi
00492919e7 Add HTTPFlow.mode to record the HTTP proxy layer mode 2016-11-12 11:58:04 +13:00
Aldo Cortesi
5be35d258f Use an enum for http protocol modes 2016-11-12 11:58:04 +13:00
Aldo Cortesi
fbaade4298 Remove promotion to raw TCP based on heuristics
This seems terribly dangerous to me. Let's expand explicit control instead.
2016-11-12 11:58:04 +13:00
Aldo Cortesi
3958940420 Test failure during 100-continue
Also:

- Remove duplicate and unused code
- Tighten scope of HttpReadDisconnect handler - we only want to ignore this for
the initial read, not for the entire block that includes things like the expect
handling.
2016-11-12 11:58:04 +13:00
Aldo Cortesi
82ac7d05a6 Bug: ask requestheaders before request body is read
Also add the beginnings of a test suite to exercise issues like this.
2016-11-12 11:58:04 +13:00
Aldo Cortesi
53b77fc475 proxy.protocol.http: cleanups, extract request validation 2016-11-12 11:58:04 +13:00
Aldo Cortesi
4eea265925 Remove unused protocol attribute on connections. 2016-11-12 11:58:04 +13:00
Aldo Cortesi
a653f314ff proxy.protocol.http: flatten for refactoring
Flatten all of _process_flow, so we can see what's going on in there.
2016-11-12 11:58:04 +13:00
Maximilian Hils
afa124a9f6 Merge pull request #1723 from Kriechi/http2-conns
http2: move h2 connection object
2016-11-09 15:19:55 +01:00
Maximilian Hils
77f05178ad mitmweb: minor fixes 2016-11-09 13:01:25 +01:00
Maximilian Hils
a0ddedff6f web: update static assets 2016-11-09 10:09:00 +01:00
Maximilian Hils
d9597add76 web: fix header update 2016-11-09 10:04:51 +01:00
Maximilian Hils
c2a130dced web: simplify flow storage 2016-11-08 17:55:22 +01:00
Maximilian Hils
85476d9915 clean up mitmweb 2016-11-08 17:54:27 +01:00
Maximilian Hils
62ca896492 fix mitmweb 2016-11-08 17:43:46 +01:00
Thomas Kriechbaumer
f140b1d84f http2: move h2 connection object 2016-11-05 20:39:42 +01:00
Aldo Cortesi
86b7661456 Remove sticky[auth,cookie] from http objects
- they're unused
- if we want them, they should be set by the addon in the new metadata store
2016-11-04 11:50:19 +13:00
Aldo Cortesi
067198a5dd Remove deprecated interfaces 2016-11-04 11:47:47 +13:00
Aldo Cortesi
4eb2b56dec Let's not over-ride __bool__ on connection objects
If I had a thousand years and every thesaurus in the world, I still couldn't
adequately express how much I dislike this piece of interface design.
2016-11-04 10:59:41 +13:00
Aldo Cortesi
deb66d3cac addons.setheaders: fix configure bug, tests to taddons 2016-11-04 09:09:39 +13:00
Aldo Cortesi
b51a96081a addons.view: test suite to 100%
This takes coverage for all of /addons to 100%
2016-11-04 08:55:01 +13:00
Aldo Cortesi
17e828b243 addons.view: test improvements 2016-11-03 20:53:06 +13:00
Aldo Cortesi
5ade93f2ad addons.streambodies: 100% test coverage 2016-11-03 20:16:07 +13:00
Aldo Cortesi
e9a96f4d7f addons.script: 100% test coverage 2016-11-03 19:36:34 +13:00
Aldo Cortesi
d31f2698a5 addons.onboarding: remove duplicate code 2016-11-03 16:20:38 +13:00
Aldo Cortesi
77e6dfe35c streamfile: make options less crazy
Instead of a (file, mode) tuple, move to "streamfile" as the path, and
"streamfile_append" to specify if we should append or not.

Leave the command-line options intact for now.
2016-11-03 15:18:55 +13:00
Aldo Cortesi
9f77c79227 FileStreamer -> StreamFile
options.wfile -> options.streamfile
2016-11-03 15:00:18 +13:00
Aldo Cortesi
43a8221989 addons: filestreamer test cov to 100%
Also address a bug in taddons that caused options changes that raised errors to
be retained rather than rolled back.
2016-11-03 14:50:23 +13:00
Aldo Cortesi
fbdce4b629 addons filstreamer: tests and bugfixes
- Move tests to taddons
- Fix a bug where options updates caused streaming to halt
2016-11-03 12:44:19 +13:00
Aldo Cortesi
b867fb35a3 addons: dumper spit and polish
- 100% test coverage
- Cleanups
- Add test/mitmproxy/addons/dumperview.py, a small utility for viewing dumper
output variations
2016-11-02 22:47:49 +13:00
Aldo Cortesi
a75b3474a4 tests: convert test_dumper to use taddons 2016-11-02 11:17:08 +13:00
Aldo Cortesi
59b269425f tests: client_playback coverage 2016-11-02 10:59:33 +13:00
Aldo Cortesi
c55e8d8f62 tests: clean up test/mitmproxy/tutils.py
Remove aliases, use mitmproxy/test/tutils instead.
2016-11-02 09:44:18 +13:00
Vincent Haupert
f9add49833 Only raise exception if an error occurred. 2016-10-31 23:53:11 +01:00
Aldo Cortesi
c33557a230 Extend mypy checks to mitmproxy/addons
Fix a number of issues to do this.
2016-11-01 10:38:35 +13:00
Aldo Cortesi
50deaf56bf Remove addons.State
- Do a first-pass port of mitmweb. Some stuff still broken.
- Remove more all State tests from the test suite
2016-11-01 10:06:28 +13:00
Aldo Cortesi
ecffaab862 console: Minor cleanups 2016-11-01 09:25:32 +13:00
Aldo Cortesi
daf355bb4c console: add caching sort keys
This is a tad complicated. The underlying sorted list implementation expects
the sort key to be stable for the lifetime of the object. However, if we sort
by size, for instance, the sort order changes as the flow progresses through
its lifecycle. We address this through two means:

- Let order keys cache the sort value by flow ID.
- Add a facility to refresh items in the list by removing and re-adding them
when they are updated.
2016-10-30 21:11:30 +13:00
Aldo Cortesi
9abfb1aac2 console: re-add follow focus 2016-10-30 19:47:32 +13:00
Aldo Cortesi
9322167eeb console: show order option in status bar 2016-10-30 18:33:51 +13:00
Aldo Cortesi
5975cc8301 console: status bar improvements
- Arrow to show flow order
- Hook statusbar up to focus change to fix some update issues
2016-10-30 18:25:33 +13:00
Aldo Cortesi
f08b57fb9b console: "v" to reverse flow order 2016-10-30 17:57:43 +13:00
Aldo Cortesi
a67a591893 console: o for flow order
"o" shortcut and --order cmdline option to change flow order.

Options is now bound to "O".
2016-10-30 17:42:34 +13:00
Aldo Cortesi
7a6bae336b addons.view: fix tests 2016-10-30 17:03:06 +13:00
Aldo Cortesi
a43cce504a console: various cleanups 2016-10-30 16:58:19 +13:00
Aldo Cortesi
62ead34a94 mitmproxy.console: Complete port to view. 2016-10-30 16:15:33 +13:00
Aldo Cortesi
6b6c44551a addons.view: Add first-class support for marking
Marking now works differently - it's no longer just another filter, it's
applied in addition to the filter. This means you can apply a filter, mark some
flows, and then toggle between marked and unmarked flows matching the filter.

I'm leaving the filter for marked flows intact - it will come in handy in other
situations.
2016-10-30 15:45:36 +13:00
Aldo Cortesi
2b76db1272 console: Port from state to view
First phase of the port - basic flow list and flow view functionality working.
More to come.
2016-10-30 14:56:36 +13:00
Aldo Cortesi
b9eb1a3479 Add options.default_contentview
This is the first step in a longer process of revamping content views. For the
moment, the option is not exposed on the command line.
2016-10-30 12:05:37 +13:00
Aldo Cortesi
944dcbaaa0 Cleanups: remove intercept from addons.state, unused imports 2016-10-30 11:13:05 +13:00
Aldo Cortesi
81a00f6f76 Replace interception in mitmproxy and mitmweb with addon
Fixes #1674
2016-10-30 11:07:09 +13:00
Aldo Cortesi
d1f14961ed test.taddon: Add cycle() method, use it to test addons.intercept 2016-10-30 10:52:57 +13:00
Aldo Cortesi
b4904d33ba Move test flow creators to mitmproxy.test.tflow 2016-10-30 10:12:39 +13:00
Aldo Cortesi
2dc3284fbb Add addons.intercept
- Add an addon to handle intercept based on a filter pattern
- Start sketching out a nicer testing truss for addons in mitmproxy.test.taddon
2016-10-29 15:44:48 +13:00
Aldo Cortesi
71d2636594 Sync 2016-10-29 12:35:10 +13:00
Aldo Cortesi
005c22445b addons.view: focus.focusflow -> focus.flow 2016-10-29 11:55:44 +13:00
Aldo Cortesi
7ecaeb0214 addons.view.focus: next and prev methods 2016-10-29 11:50:10 +13:00
Aldo Cortesi
32a0a7b860 addons.view: flow settings
Add a flow settings mechanism, enable focus and settings unilaterally.
2016-10-29 11:37:32 +13:00
Aldo Cortesi
14df969434 addons.view.focus: Better handling of view refresh
When we refresh and our current focus goes out of scope, we set the focus to
the element nearest the old focus.
2016-10-29 11:08:35 +13:00
Aldo Cortesi
90e7142b5c addons.View: better order reversal
Deal with some subtleties in order reversal, add a toggle method that emits
refresh.
2016-10-29 10:55:57 +13:00
Aldo Cortesi
12a70d03ad addons.view: Add a focus tracker 2016-10-29 10:17:29 +13:00
Aldo Cortesi
9dcc3a3e20 addons.View: hook up signals 2016-10-29 08:25:19 +13:00
Aldo Cortesi
69bacee1d8 Sketch out addons.View
The first iteration of a replacement for addons.State
2016-10-29 08:25:19 +13:00
Aldo Cortesi
9be34baa40 Merge pull request #1687 from Kriechi/http2-type-information
http2: add more type information
2016-10-29 08:12:41 +13:00
Thomas Kriechbaumer
d86cb76e5b http2: add push indicator 2016-10-27 20:33:30 -07:00
Thomas Kriechbaumer
f1878eb051 http2: add more type information 2016-10-27 20:24:57 -07:00
Maximilian Hils
e71b634c58 Merge pull request #1684 from mhils/1675
Fix #1675
2016-10-27 16:22:37 -07:00
Maximilian Hils
4ac4fe2849 Merge pull request #1671 from mitmproxy/bump-h2
Bump h2 version
2016-10-27 16:13:30 -07:00
Maximilian Hils
62c9c3db4f fix #1675 2016-10-27 16:12:18 -07:00
Thomas Kriechbaumer
44ac370f08 upgrade h2 to >=2.5.0 2016-10-27 15:10:53 -07:00
Maximilian Hils
48f51849b9 fix #1676 2016-10-27 13:10:57 -07:00
Maximilian Hils
a93baad655 Merge pull request #1680 from mhils/remove-lrucache
Remove mitmproxy.utils.lrucache
2016-10-27 11:45:35 -07:00
Maximilian Hils
97f1236c99 Merge pull request #1670 from chhsiao90/display-cipher-#582
Resolved #582: display ClientConnection select cipher of TLS
2016-10-27 11:38:22 -07:00
Maximilian Hils
6b4c705197 remove mitmproxy.utils.lrucache 2016-10-27 11:24:36 -07:00
Aldo Cortesi
72ac572226 Merge pull request #1673 from mhils/issue-1620
fix #1620
2016-10-27 08:26:48 +13:00
chhsiao90
d52f35428c Show tls version on console 2016-10-26 14:36:14 +08:00
Maximilian Hils
eda1b39a74 minor fixes 2016-10-25 22:06:52 -07:00
Maximilian Hils
f11b289c39 fix #1620 2016-10-25 21:56:13 -07:00
Maximilian Hils
b1bdae3d1c typecheck options, fix current inconsistencies 2016-10-25 20:45:48 -07:00
chhsiao90
960f2e8bf0 Resolved #582: display ClientConnection select cipher of TLS 2016-10-26 11:32:42 +08:00
Maximilian Hils
917c701562 make options keyword-only 2016-10-25 17:34:30 -07:00
Maximilian Hils
145c2892f7 Merge pull request #1664 from chhsiao90/sni-display-#1639
Resolved #1639: display sni on ClientConnection
2016-10-24 19:56:05 -07:00
chhsiao90
39ac29e37c Resolved #1639: display sni on ClientConnection 2016-10-25 10:46:53 +08:00
Maximilian Hils
ef4e9b2b85 Merge pull request #1656 from mhils/improve-export-2
Improve Flow Export
2016-10-24 19:19:58 -07:00
Maximilian Hils
21f133fae9 fix FlowView._get_content_view 2016-10-24 16:03:28 -07:00
Sahil Chelaramani
7b3505336a replace mitmproxy.utils.lrucache with functools.lru_cache 2016-10-24 16:03:16 -07:00
Maximilian Hils
c14ae74d2e disable erroneous linting errors 2016-10-24 15:59:57 -07:00
Maximilian Hils
5670e61a31 add version check for Python 3.4- 2016-10-24 14:26:07 -07:00
Maximilian Hils
e87daa70f3 improve flow export 2016-10-22 20:32:39 -07:00
Aldo Cortesi
6170493615 Don't call error for flow kill
This is now the error handler on master, so whatever the intetion was it's now
definitely wrong.
2016-10-22 15:56:42 +13:00
Aldo Cortesi
9fc1547053 console: give statusbar messages a default timeout 2016-10-22 15:06:24 +13:00
Aldo Cortesi
cc8b422d9d multidict: remove to_dict
We never use it, and it is dangerously ambiguous when a key is associated with
a list.
2016-10-21 11:42:21 +13:00
Aldo Cortesi
18ee6255c0 multidict: ditch ImmutableMultiDict
A contorted class we only use for cookie attributes. We don't need it.
2016-10-21 11:40:05 +13:00
Aldo Cortesi
635c77d4ed console: correct log event handling i
Fixes #1640
2016-10-21 09:25:34 +13:00