Grasscutters/mitmproxy
3
0
Fork 0
mirror of https://github.com/Grasscutters/mitmproxy.git synced 2024-11-27 02:24:18 +00:00
Code Issues Projects Releases Wiki Activity
1,064 Commits 1 Branch 67 Tags 50 MiB
f47d89ff4e
Commit Graph

797 Commits

Author SHA1 Message Date
Maximilian Hils
ba47690a03 always read files in binary mode 2013-06-16 00:23:44 +02:00
Maximilian Hils
2b4af8d475 add support for multiple scripts and script arguments. refs #76 2013-06-13 16:09:38 +02:00
Aldo Cortesi
d3beaa7382 Merge pull request #132 from ipopov/master 
A humble pull request
 2013-06-08 16:28:47 -07:00
Aldo Cortesi
1a5c27aa7d Massage content-type before sending it to mime detection 
Fixes #67
 2013-06-09 11:26:44 +12:00
Aldo Cortesi
7ef68b5a13 Fix creation of new response when none existed before. 
Fixes #133
 2013-06-09 11:14:34 +12:00
Ivaylo Popov
ffeede9b39 Use lsof instead of pfctl to find target host on OSX in transparent mode. 2013-05-27 23:09:42 -04:00
Michael Bisbjerg
125b3e5e5b - Quick-fix for issue #128 
New bug: It correct-cases Content-Length for any webserver sending other casings, like CONTENT-LENGTH.
 2013-05-21 15:57:14 +02:00
Aldo Cortesi
bc88930fb7 Merge branch 'master' of ssh.github.com:cortesi/mitmproxy 2013-05-05 13:19:14 +12:00
Aldo Cortesi
9fa09cc1f9 Fix crash in client playback. 2013-05-05 13:18:52 +12:00
Aldo Cortesi
5cd7563d12 Minor coverage. 2013-04-30 09:13:33 +12:00
Jason A. Novak
f78dada550 Add error checking to ViewProtobuf 
There are protobufs that protoc can't parse.  When protoc --decode_raw
fails, it returns nothing to stdin, and writes "Failed to parse input."
to stderr. Before this commit, if protoc --decode_raw couldn't parse
the protobuf, the blank stdout output would get returned to the view;
with this commit stderr gets caught and returned to the view.
 2013-04-21 12:46:37 -05:00
Aldo Cortesi
61c794e08f Merge pull request #107 from rouli/master 
Adding remote TCP and SSL setup timestamps
 2013-04-19 17:19:26 -07:00
Aldo Cortesi
793c41a5c4 Merge pull request #112 from hamstah/protobuf-view 
Adds a new view for protocol buffers
 2013-04-19 17:18:44 -07:00
Alexis Hildebrandt
3d7f31b23d Correct display mode highlight keys 
Add html display mode to the help documentation.
Correct html and hex display mode highlight keys (help used 'h' for hex).
Correct json display mode highlight keys.
 2013-04-16 23:54:34 +02:00
Nicolas Esteves
d4cfbbb822 Adds a new view for protocol buffers 
The view uses protoc from the Google protocol buffer
tools. If the tool isn't installed, the view isn't
shown.

Google protobuf repo:
https://code.google.com/p/protobuf/
 2013-04-06 19:21:13 +01:00
Aldo Cortesi
51b775cfd4 Merge pull request #101 from eentzel/keep-blank-params 
Keep blank URL parameters
 2013-04-04 15:55:51 -07:00
Aldo Cortesi
ca9c60d2eb Docs. 2013-04-05 11:55:28 +13:00
Aldo Cortesi
e3fd0e838d Add a basic built-in web app. 2013-03-25 09:20:26 +13:00
Aldo Cortesi
98e4421a90 Trim docs. 2013-03-23 15:42:25 +13:00
Aldo Cortesi
800af34763 Fix crash on intercept. 
Fixes #106
 2013-03-23 14:40:03 +13:00
Rouli
c6bf28f3f7 adding tcp and ssl setup timestamps to get better resolution on flows performance 2013-03-19 18:21:52 +02:00
Rouli
c94aadcb0e Merge remote-tracking branch 'upstream/master' 2013-03-18 14:24:13 +02:00
Aldo Cortesi
6614498744 Update styling, GameCenter highscore tutorial. 2013-03-18 08:36:56 +13:00
Aldo Cortesi
d2d3eb6490 Un-break unit tests. Tsk tsk. 2013-03-17 17:53:48 +13:00
Aldo Cortesi
e50da8164f Enable --host option for mitmdump 2013-03-17 17:43:31 +13:00
Aldo Cortesi
0e993bec6f Add the --host option, which uses the value in the Host header for dispaly URLs. 
- Can be toggled with "o" then "h" in mitmproxy
- Useful for transparent mode
 2013-03-17 17:37:54 +13:00
Aldo Cortesi
790ad468e4 Fix bug that caused mis-identification of some HTTPS connections in transparent mode. 2013-03-17 14:35:36 +13:00
Aldo Cortesi
cfb5ba89ce Introduce a filtered flow writer, and use it in dump.py 
Fixes #104
 2013-03-14 09:19:43 +13:00
Aldo Cortesi
cde66cd584 Fuzzing, and fixes for errors found with fuzzing. 2013-03-03 22:03:27 +13:00
Aldo Cortesi
7835e0c2c7 Begin some simple fuzzing with pathod. 
Finally doing what I started writing pathod for in the first place...
 2013-03-03 14:56:56 +13:00
Aldo Cortesi
e608d10f45 Remove __slots__ to make it possible to inherit from Options classes. 2013-03-03 12:26:20 +13:00
Aldo Cortesi
75b5c97095 Revert "show current filepath in status bar" 
This reverts commit bf8367d6cf.

This just doesn't work. We need a better solution, probably in the next release.
 2013-03-03 12:18:19 +13:00
Aldo Cortesi
2465b8a376 100% unit test coverage on proxy.py. Hallelujah! 2013-03-03 12:13:33 +13:00
Aldo Cortesi
d5876a12ed Unit test proxy option parsing. 2013-03-03 11:58:57 +13:00
Aldo Cortesi
5c6587d4a8 Move HTTP auth module to netlib. 2013-03-03 10:37:06 +13:00
Aldo Cortesi
c20d1d7d32 Extend unit tests for proxy.py to some tricky cases. 2013-03-02 22:42:36 +13:00
Aldo Cortesi
415844511c Test cert generation errors. 2013-03-02 16:59:16 +13:00
Aldo Cortesi
a95d78438c Test SNI for transparent mode. 2013-03-02 15:06:49 +13:00
Aldo Cortesi
10db82e9a0 Test SNI for ordinary proxy connections. 2013-03-02 14:52:05 +13:00
Aldo Cortesi
ba674ad551 New SNI handling mechanism. 2013-03-01 09:05:39 +13:00
Rouli
b6cae7cd2d Merge remote-tracking branch 'upstream/master' 2013-02-28 13:28:57 +02:00
Rouli
35f36481b9 adding __str__ to make export to har nicer 2013-02-28 13:28:42 +02:00
Aldo Cortesi
0257815141 Significantly simplify server connection handling, and test. 2013-02-24 22:24:21 +13:00
Aldo Cortesi
705559d65e Refactor to prepare for SNI fixes. 2013-02-24 17:35:24 +13:00
Aldo Cortesi
d0639e8925 Handle server disconnects better. 
Server connections can be closed for legitimate reasons, like timeouts. If
we've already pumped data over a server connection, we reconnect on error. If
not, we treat it as a legitimate error and pass it on to the client.

Fixes #85
 2013-02-24 14:04:56 +13:00
Aldo Cortesi
05e4d4468e Test request and response kill functionality. 2013-02-23 21:59:25 +13:00
Aldo Cortesi
269780c577 Unit test dummy response functions. 2013-02-23 16:34:59 +13:00
Aldo Cortesi
f203881b0d Remove redundant clause in controller.Reply 2013-02-23 14:13:43 +13:00
Aldo Cortesi
7800b7c910 Refactor proxy core communications to be clearer. 2013-02-23 14:10:27 +13:00
Aldo Cortesi
aaf892e3af Significantly refactor the master/slave message passing interface. 2013-02-17 12:42:48 +13:00
Eric Entzel
6bcf29c0ed Keep blank URL parameters 
TODO: This should probably be configurable
 2013-02-11 13:22:25 +11:00
Aldo Cortesi
782bbee8c0 Unit tests for ServerConnectionPool 2013-01-29 11:35:57 +13:00
Aldo Cortesi
2aa175a6ca Stub implementation of a server connection pool. 2013-01-29 10:55:19 +13:00
Aldo Cortesi
6f157d936f Merge pull request #99 from rouli/master 
Adding helper functions to make HAR export easier
 2013-01-28 13:28:49 -08:00
Rouli
330fbfe8cc adding helper functions to make HAR export easier 2013-01-28 17:37:25 +02:00
Aldo Cortesi
a74ca40660 Unravel enormously long read_request into three distinct methods. 2013-01-28 22:26:25 +13:00
Aldo Cortesi
57f01ffb07 Test suite, remove extraneous code. 2013-01-28 21:59:03 +13:00
phil plante
68952d579e Force flush of file content in dump 
The dump file would be end up corrupted sometimes when working with mitmdump in a VM.  Adding an explicit flush seems to have resolved the file sync issues.
 2013-01-24 20:46:50 -08:00
Aldo Cortesi
25cb9471f0 Add tests for client certificate support. 2013-01-20 22:39:28 +13:00
Aldo Cortesi
294bca139c Merge branch 'master' of ssh.github.com:cortesi/mitmproxy 2013-01-19 17:07:27 +13:00
Chris Neasbitt
e9264a8253 Fixed a bug in format_flow in common.py. Changed the reference from timestamp to timestamp_start. 2013-01-18 23:04:11 -05:00
Aldo Cortesi
6600c589ab Rudimentary testing for client certs. 2013-01-18 17:08:30 +13:00
Aldo Cortesi
7a79eeb143 Merge branch 'master' of ssh.github.com:cortesi/mitmproxy 
Conflicts:
	test/test_server.py
 2013-01-18 14:50:31 +13:00
Rouli
446f9f0a0f Merge remote-tracking branch 'upstream/master' 2013-01-17 17:33:29 +02:00
Rouli
20fa6a3083 changing requests and responses to have two timestamps, one marking their initiation, and the other their complete 2013-01-17 17:32:56 +02:00
Aldo Cortesi
d0ee4d60d0 Unit tests and minor code refactoring for ServerConnection. 2013-01-05 19:44:12 -08:00
Aldo Cortesi
060e3198bc Remove cert_wait_time flag. 
We now cater for this by generating certs with a commencement date an hour in
the past in netlib.
 2013-01-06 01:18:47 +13:00
Aldo Cortesi
891c441a6d Use new netlib certificate store implementation. 2013-01-06 01:16:08 +13:00
Aldo Cortesi
9cfc785cd3 Unit test love - 100% for flow.py, dump.py 2013-01-05 21:56:33 +13:00
Aldo Cortesi
46ab6ed491 Minor cleanups of proxy request handling. 2013-01-04 14:19:32 +13:00
Aldo Cortesi
09f664cdea Refactor proxy auth a bit 
- Remove authentication scheme option. We only support basic at the moment -
we'll add the option back when we diversify.
- Add some meta variables to make printout nicer
 2013-01-02 17:35:44 +13:00
Aldo Cortesi
7b3d3dc85e Documentation, setup.py updates, styling. 2013-01-02 14:02:41 +13:00
Aldo Cortesi
e42136a6ef Better error handling for transparent mode remote address resolution. 2013-01-01 11:24:11 +13:00
Aldo Cortesi
e2dc7ba09d First draft of OSX transparent proxy mode. 2013-01-01 11:13:56 +13:00
Aldo Cortesi
5347cb9c26 More work on proxy auth 
- Strip auth header if auth succeeds, so it's not passed upstream
- Actually use realm specification to BasicProxyAuth, and make it mandatory
- Cleanups and unit tests
 2012-12-31 10:56:44 +13:00
Aldo Cortesi
3b84111493 Test and robustify BasicProxyAuth.parse_auth_value 
- This is partly in preparation for moving the implementation to netlib
- Also add an unparse_auth_value for testing (and use in pathod once the move is done)
 2012-12-31 10:34:25 +13:00
Aldo Cortesi
018c229ae4 Start solidifying proxy authentication 
- Add a unit test file
- Remove some extraneous methods
- Change the auth API to make the authenticate method take a header object.
 2012-12-31 09:15:56 +13:00
israel
440a9f6bda adding some simple authetication code to limit proxy access 2012-12-30 01:41:58 -08:00
israel
935505bc4f adding some simple authetication code to limit proxy access 2012-12-30 01:24:30 -08:00
Aldo Cortesi
3c8dcf8808 Merge pull request #82 from kanzure/show-filepath-in-statusbar 
Show current filepath in status bar
 2012-12-30 00:49:45 -08:00
Bryan Bishop
bf8367d6cf show current filepath in status bar 
Showing the filename is useful when looking at multiple .mitm files
simultaneously.
 2012-12-26 22:14:39 -06:00
Bryan Bishop
4d250095cb fix external viewer using shlex 
This makes spawn_external_viewer not crash when $EDITOR or $PAGER have
spaces or multiple arguments.

In addition, spawn_external_viewer now chmods the file to read-only to
remind users who use only an $EDITOR that this function does not read
the file when the user returns.

Also, some of the redundant exception case handling for editing has been
consolidated.

fixes #79
 2012-12-22 18:26:15 -06:00
Bryan Bishop
1c6139e013 remove trailing whitespace 2012-12-09 15:57:11 -06:00
Bryan Bishop
505da188eb Show an error when $EDITOR/$PAGER are unset. 
This catches an exception that otherwise crashes mitmproxy.

fixes cortesi/mitmproxy#71
 2012-12-05 12:58:29 -08:00
Aldo Cortesi
8c976ac7f0 Substantially rewrite AMF decoding. 
This is tricky, but we should now handle a lot more corner-cases.
 2012-11-26 13:25:07 +13:00
Mathieu Mitchell
15c367ffb4 Workaround for PIL's ambiguious import method and/or easy_install PIL packaging problem. 
PIL documents two different way to import it's modules:
* import Image (http://www.pythonware.com/library/pil/handbook/introduction.htm)
* from PIL import Image (http://www.pythonware.com/library/pil/handbook/image.htm)

The same problem was noted in Django at https://code.djangoproject.com/ticket/6054
 2012-11-23 11:48:24 -05:00
Aldo Cortesi
626fc39804 Move eventlog to new Urwid container API. 2012-11-23 15:44:43 +13:00
Aldo Cortesi
57d6650e8e Urwid 1.1 compatibility. 2012-10-29 09:30:59 +13:00
Aldo Cortesi
68f1000e42 Improve error reporting for one-shot scripts. 2012-10-11 11:12:06 +13:00
Aldo Cortesi
0d59fd7e01 Move cleanBin and hexdump into netutils. 2012-09-24 11:21:12 +12:00
Aldo Cortesi
21f74efa10 Stub out ctypes structures for OSX transparent mode. 2012-09-17 11:05:20 +12:00
Aldo Cortesi
d115b5ae70 Expand Flow.match to accept either a string or a compiled filter expression. 2012-09-14 09:41:01 +12:00
Aldo Cortesi
54cee9db7f Catch a potential exception on connection finalization. 2012-09-14 09:40:13 +12:00
Aldo Cortesi
b7d89f6919 Don't run replace or header hooks on error. 2012-09-02 12:57:49 +12:00
Aldo Cortesi
5630d3f660 Add help entry for H global header shortcut. 2012-09-02 11:53:00 +12:00
Aldo Cortesi
9c009a872e Add U shortcut to add user-agent strings to global Headers editor. 2012-09-02 11:50:17 +12:00
Aldo Cortesi
3fc9af63c1 Fix a crash when re-editing a path prompt after an error. 2012-08-31 13:41:08 +12:00
Aldo Cortesi
26cefc95e8 Make grid editor file reading more robust. 2012-08-31 13:33:18 +12:00
Aldo Cortesi
a33e90f081 Add a shortcut to header editor to add standard User-Agent strings. 2012-08-31 13:28:04 +12:00
Aldo Cortesi
71ae158d7b Display "No Content" instead of a parse error when there is no content. 2012-08-30 12:51:04 +12:00
Aldo Cortesi
a2f9ca1d4d Add application/javascript to ~a filter asset matcher. 2012-08-30 12:42:31 +12:00
Aldo Cortesi
38ddbcc314 Add a snippet to GridEditor help text explaining that we're using escaped strings. 2012-08-25 13:29:05 +12:00
Aldo Cortesi
87463049f1 Add a "R" shortcut to GridEditor, letting the user read unescaped data from file. 2012-08-25 13:25:59 +12:00
Aldo Cortesi
514e19b172 Do away with explicit encodings, and display an error message for invalid values. 2012-08-25 13:13:16 +12:00
Aldo Cortesi
c6d1fe9e59 Use Python-style escaped strings in GridEditor. 2012-08-25 12:54:34 +12:00
Aldo Cortesi
82893ffae2 Add an "r" shortcut in grid editors to read value from file. 2012-08-25 12:21:45 +12:00
Aldo Cortesi
3787f8befb Add a graphical editor for path components (e -> p shortcut from request view screen). 2012-08-19 13:15:54 +12:00
Aldo Cortesi
a77ccc406d Getter and setter for path component on Requests. 2012-08-19 13:03:21 +12:00
Aldo Cortesi
1b7990897e Command-line options for header setting. 2012-08-19 00:14:16 +12:00
Aldo Cortesi
3e96015e61 Add SetHeaders, analogous to ReplaceHooks, with a graphical editor in mitmproxy (H shortcut). 
SetHeaders defines headers that are set on flows, based on a matching pattern.
Existing headers are over-ridden.
 2012-08-18 23:41:04 +12:00
Aldo Cortesi
53e453f72e Use the new ODict get_first convenience function in a bunch of places. 2012-08-18 18:14:30 +12:00
Aldo Cortesi
15e234558d Further content view cleaups. 2012-08-18 17:51:34 +12:00
Aldo Cortesi
b70e91bbd4 Send tracebacks from content viewers to event log. 
Also, 100% test coverage for content viewers.
 2012-08-18 17:42:40 +12:00
Aldo Cortesi
e8553f966f Further simplifcation and testing of contentviews. 2012-08-18 17:29:29 +12:00
Aldo Cortesi
11c63dcb9f Huge cleanup of content viewers. 2012-08-18 17:08:17 +12:00
Sahn Lam
3189d144a5 Optional AMF decoding support 
If PyAMF is installed, enable AMF decoding.
 2012-08-17 18:45:26 -07:00
Maximilian Hils
b0566b9d4c add dummy cert dir 2012-08-17 19:13:56 +02:00
Maximilian Hils
ed389d8f05 use argparse instead of optparse 2012-08-17 19:11:59 +02:00
Aldo Cortesi
bbaa8bdba5 Add an HTML text outline view. 
Uses html2text, from here:

https://github.com/aaronsw/html2text
 2012-08-18 01:37:30 +12:00
Aldo Cortesi
1bfe847a84 Stop server playback after current playback buffer is exhausted. 2012-08-18 00:23:41 +12:00
Aldo Cortesi
20b270ae9a Server replay from current buffer. 2012-08-18 00:13:04 +12:00
Jim Lloyd
0ef18a7cba Adds --dummy-certs option to specify certdir 
If --dummy-certs=CERTSDIR is provided, use CERTSDIR as the location
for generating/finding the dummy certs. And in this case, preserve
the CERTSDIR directory on exit.
 2012-08-06 14:09:35 -07:00
Aldo Cortesi
32ad26f8bf Add a size() method to flow.Request and flow.Response. 2012-08-04 13:18:05 +12:00
Rob Wills
8e68426ad6 Return from Flow View behaviour 
Following focus trumps the re-focusing the "current" (or just-viewed) flow.
 2012-08-02 00:06:34 -07:00
Rob Wills
c985e22196 Press "F" to Follow 
Following of the flow list
- ConsoleState now has a follow_focus property
-- when True it will cause focus to follow added flows
- ConsoleMaster implements toggle_follow_flows()
-- when enabling follow, also jumps to most recent flow
- StatusBar reports follow_focus state
-- adds "following" to option list when true
- Added "F" to FlowList keys
-- implementation requests ConsoleMaster to toggle_follow_flows()
 2012-08-01 23:57:56 -07:00
Aldo Cortesi
87d05a95ff Handle invalid headers. 2012-07-30 12:54:50 +12:00
Aldo Cortesi
bb124e23b2 Prevent mitmproxy from treating logged info as urwid markup. 2012-07-27 10:08:10 +12:00
Aldo Cortesi
47b5fd666d Fix mitmproxy file reading. 2012-07-27 02:13:21 +12:00
Aldo Cortesi
31a092f6b4 Minor refactoring. Make stop_stream also close the associated file descriptor. 2012-07-27 00:19:18 +12:00
Aldo Cortesi
f93a621856 Only log real errors in WSGI apps. 2012-07-24 16:18:22 +12:00
Aldo Cortesi
b4e9e55c34 Be more tolerant of corrupted or truncated flows. 
We load as far as possible. mitmproxy will only terminate if it was not able to
recover any flows. mitmdump will stop loading as soon as an error is
encountered, but not exit with an error.
 2012-07-24 15:15:41 +12:00
Maximilian Hils
c643234c98 fix -r option, read file in binary mode 2012-07-24 03:11:28 +02:00
Chris Neasbitt
525a8f6a16 Fixed a bug causing an AttributeError when request is set to false but response not None in ProxyHandler.handle_request 2012-07-17 13:24:15 -04:00
Aldo Cortesi
e4079aa746 Add an ~a filter expression, matching an asset content type in responses. 
Asset content types are Javascript, images, Flash and CSS. This is useful
because doing a quick "!~a" while auditing an app will filter out the majority
of the static asset cruft, letting you focus on what matters.
 2012-07-14 16:55:21 +12:00
Aldo Cortesi
150814f6a8 Jump back to correct flow when limit is applied. 2012-07-14 15:22:44 +12:00
Aldo Cortesi
a44a76a7da Merge branch 'master' of github.com:cortesi/mitmproxy 2012-07-11 07:17:20 +12:00
Aldo Cortesi
1d09a558a7 Fix a subtle termination condition when there's an error in a WSGI app. 2012-07-11 07:16:06 +12:00
Aldo Cortesi
01b8b0d876 Refine semantics of replay_request method. 2012-07-10 23:29:33 +12:00
Aldo Cortesi
04d9ec8c3c Make WSGI apps work in transparent mode. 2012-07-10 15:53:53 +12:00
Aldo Cortesi
79af9e89c4 Test replay corner cases. Fix discovered bugs. 2012-07-09 11:18:03 +12:00
Aldo Cortesi
aab45078ad Unit test request replay thread. 
This is a small patch, but is the culmination of lots of work: we can now unit
test the deep innards of mitmproxy, with coverage. There's a lot more to come
in this vein.
 2012-07-09 11:03:55 +12:00
Aldo Cortesi
3749d52b66 Make mitmproxy "W" over-write, not append. 
The -w options in mitmdump and mitproxy should do the same thing. Append can
have unexpected consequences if the existing file is not a mitmdump.
 2012-07-09 10:28:01 +12:00
Aldo Cortesi
729fd9301f Use FlowMaster stream from mitmdump. 2012-07-09 10:26:45 +12:00
Aldo Cortesi
bbd9acf551 Use FlowMaster streaming for "W" in mitmproxy. 2012-07-09 10:22:14 +12:00
Aldo Cortesi
572e8a4962 Add streaming to FlowMaster 2012-07-09 10:18:37 +12:00
Aldo Cortesi
097b566e54 Handle new netlib.tcp.NetLibDisconnect exception. 2012-07-08 23:49:44 +12:00
Aldo Cortesi
ceb12438b6 Fix minor error in filter help. 2012-07-06 22:33:09 +12:00
Aldo Cortesi
d02bcade3a Add a domain match filter (~d regex) 2012-07-06 22:21:44 +12:00
Aldo Cortesi
c4426952ad Protect against exceptions thrown by third-party view libraries. 2012-07-06 15:43:33 +12:00
Aldo Cortesi
a7e64a1a03 mitmproxy: "W" shortcut key streams flows to file as responses arrive. 2012-07-06 14:41:10 +12:00