Grasscutters/mitmproxy
3
0
Fork 0
mirror of https://github.com/Grasscutters/mitmproxy.git synced 2024-11-27 18:31:22 +00:00
Code Issues Projects Releases Wiki Activity
1,216 Commits 1 Branch 67 Tags 50 MiB
a48cccadb5
Commit Graph

911 Commits

Author SHA1 Message Date
Aldo Cortesi
415844511c Test cert generation errors. 2013-03-02 16:59:16 +13:00
Aldo Cortesi
a95d78438c Test SNI for transparent mode. 2013-03-02 15:06:49 +13:00
Aldo Cortesi
10db82e9a0 Test SNI for ordinary proxy connections. 2013-03-02 14:52:05 +13:00
Aldo Cortesi
ba674ad551 New SNI handling mechanism. 2013-03-01 09:05:39 +13:00
Rouli
b6cae7cd2d Merge remote-tracking branch 'upstream/master' 2013-02-28 13:28:57 +02:00
Rouli
35f36481b9 adding __str__ to make export to har nicer 2013-02-28 13:28:42 +02:00
Aldo Cortesi
0257815141 Significantly simplify server connection handling, and test. 2013-02-24 22:24:21 +13:00
Aldo Cortesi
705559d65e Refactor to prepare for SNI fixes. 2013-02-24 17:35:24 +13:00
Aldo Cortesi
d0639e8925 Handle server disconnects better. 
Server connections can be closed for legitimate reasons, like timeouts. If
we've already pumped data over a server connection, we reconnect on error. If
not, we treat it as a legitimate error and pass it on to the client.

Fixes #85
 2013-02-24 14:04:56 +13:00
Aldo Cortesi
05e4d4468e Test request and response kill functionality. 2013-02-23 21:59:25 +13:00
Aldo Cortesi
269780c577 Unit test dummy response functions. 2013-02-23 16:34:59 +13:00
Aldo Cortesi
f203881b0d Remove redundant clause in controller.Reply 2013-02-23 14:13:43 +13:00
Aldo Cortesi
7800b7c910 Refactor proxy core communications to be clearer. 2013-02-23 14:10:27 +13:00
Aldo Cortesi
aaf892e3af Significantly refactor the master/slave message passing interface. 2013-02-17 12:42:48 +13:00
Eric Entzel
6bcf29c0ed Keep blank URL parameters 
TODO: This should probably be configurable
 2013-02-11 13:22:25 +11:00
Aldo Cortesi
782bbee8c0 Unit tests for ServerConnectionPool 2013-01-29 11:35:57 +13:00
Aldo Cortesi
2aa175a6ca Stub implementation of a server connection pool. 2013-01-29 10:55:19 +13:00
Aldo Cortesi
6f157d936f Merge pull request #99 from rouli/master 
Adding helper functions to make HAR export easier
 2013-01-28 13:28:49 -08:00
Rouli
330fbfe8cc adding helper functions to make HAR export easier 2013-01-28 17:37:25 +02:00
Aldo Cortesi
a74ca40660 Unravel enormously long read_request into three distinct methods. 2013-01-28 22:26:25 +13:00
Aldo Cortesi
57f01ffb07 Test suite, remove extraneous code. 2013-01-28 21:59:03 +13:00
phil plante
68952d579e Force flush of file content in dump 
The dump file would be end up corrupted sometimes when working with mitmdump in a VM.  Adding an explicit flush seems to have resolved the file sync issues.
 2013-01-24 20:46:50 -08:00
Aldo Cortesi
25cb9471f0 Add tests for client certificate support. 2013-01-20 22:39:28 +13:00
Aldo Cortesi
294bca139c Merge branch 'master' of ssh.github.com:cortesi/mitmproxy 2013-01-19 17:07:27 +13:00
Chris Neasbitt
e9264a8253 Fixed a bug in format_flow in common.py. Changed the reference from timestamp to timestamp_start. 2013-01-18 23:04:11 -05:00
Aldo Cortesi
6600c589ab Rudimentary testing for client certs. 2013-01-18 17:08:30 +13:00
Aldo Cortesi
7a79eeb143 Merge branch 'master' of ssh.github.com:cortesi/mitmproxy 
Conflicts:
	test/test_server.py
 2013-01-18 14:50:31 +13:00
Rouli
446f9f0a0f Merge remote-tracking branch 'upstream/master' 2013-01-17 17:33:29 +02:00
Rouli
20fa6a3083 changing requests and responses to have two timestamps, one marking their initiation, and the other their complete 2013-01-17 17:32:56 +02:00
Aldo Cortesi
d0ee4d60d0 Unit tests and minor code refactoring for ServerConnection. 2013-01-05 19:44:12 -08:00
Aldo Cortesi
060e3198bc Remove cert_wait_time flag. 
We now cater for this by generating certs with a commencement date an hour in
the past in netlib.
 2013-01-06 01:18:47 +13:00
Aldo Cortesi
891c441a6d Use new netlib certificate store implementation. 2013-01-06 01:16:08 +13:00
Aldo Cortesi
9cfc785cd3 Unit test love - 100% for flow.py, dump.py 2013-01-05 21:56:33 +13:00
Aldo Cortesi
46ab6ed491 Minor cleanups of proxy request handling. 2013-01-04 14:19:32 +13:00
Aldo Cortesi
09f664cdea Refactor proxy auth a bit 
- Remove authentication scheme option. We only support basic at the moment -
we'll add the option back when we diversify.
- Add some meta variables to make printout nicer
 2013-01-02 17:35:44 +13:00
Aldo Cortesi
7b3d3dc85e Documentation, setup.py updates, styling. 2013-01-02 14:02:41 +13:00
Aldo Cortesi
e42136a6ef Better error handling for transparent mode remote address resolution. 2013-01-01 11:24:11 +13:00
Aldo Cortesi
e2dc7ba09d First draft of OSX transparent proxy mode. 2013-01-01 11:13:56 +13:00
Aldo Cortesi
5347cb9c26 More work on proxy auth 
- Strip auth header if auth succeeds, so it's not passed upstream
- Actually use realm specification to BasicProxyAuth, and make it mandatory
- Cleanups and unit tests
 2012-12-31 10:56:44 +13:00
Aldo Cortesi
3b84111493 Test and robustify BasicProxyAuth.parse_auth_value 
- This is partly in preparation for moving the implementation to netlib
- Also add an unparse_auth_value for testing (and use in pathod once the move is done)
 2012-12-31 10:34:25 +13:00
Aldo Cortesi
018c229ae4 Start solidifying proxy authentication 
- Add a unit test file
- Remove some extraneous methods
- Change the auth API to make the authenticate method take a header object.
 2012-12-31 09:15:56 +13:00
israel
440a9f6bda adding some simple authetication code to limit proxy access 2012-12-30 01:41:58 -08:00
israel
935505bc4f adding some simple authetication code to limit proxy access 2012-12-30 01:24:30 -08:00
Aldo Cortesi
3c8dcf8808 Merge pull request #82 from kanzure/show-filepath-in-statusbar 
Show current filepath in status bar
 2012-12-30 00:49:45 -08:00
Bryan Bishop
bf8367d6cf show current filepath in status bar 
Showing the filename is useful when looking at multiple .mitm files
simultaneously.
 2012-12-26 22:14:39 -06:00
Bryan Bishop
4d250095cb fix external viewer using shlex 
This makes spawn_external_viewer not crash when $EDITOR or $PAGER have
spaces or multiple arguments.

In addition, spawn_external_viewer now chmods the file to read-only to
remind users who use only an $EDITOR that this function does not read
the file when the user returns.

Also, some of the redundant exception case handling for editing has been
consolidated.

fixes #79
 2012-12-22 18:26:15 -06:00
Bryan Bishop
1c6139e013 remove trailing whitespace 2012-12-09 15:57:11 -06:00
Bryan Bishop
505da188eb Show an error when $EDITOR/$PAGER are unset. 
This catches an exception that otherwise crashes mitmproxy.

fixes cortesi/mitmproxy#71
 2012-12-05 12:58:29 -08:00
Aldo Cortesi
8c976ac7f0 Substantially rewrite AMF decoding. 
This is tricky, but we should now handle a lot more corner-cases.
 2012-11-26 13:25:07 +13:00
Mathieu Mitchell
15c367ffb4 Workaround for PIL's ambiguious import method and/or easy_install PIL packaging problem. 
PIL documents two different way to import it's modules:
* import Image (http://www.pythonware.com/library/pil/handbook/introduction.htm)
* from PIL import Image (http://www.pythonware.com/library/pil/handbook/image.htm)

The same problem was noted in Django at https://code.djangoproject.com/ticket/6054
 2012-11-23 11:48:24 -05:00
Aldo Cortesi
626fc39804 Move eventlog to new Urwid container API. 2012-11-23 15:44:43 +13:00
Aldo Cortesi
57d6650e8e Urwid 1.1 compatibility. 2012-10-29 09:30:59 +13:00
Aldo Cortesi
68f1000e42 Improve error reporting for one-shot scripts. 2012-10-11 11:12:06 +13:00
Aldo Cortesi
0d59fd7e01 Move cleanBin and hexdump into netutils. 2012-09-24 11:21:12 +12:00
Aldo Cortesi
21f74efa10 Stub out ctypes structures for OSX transparent mode. 2012-09-17 11:05:20 +12:00
Aldo Cortesi
d115b5ae70 Expand Flow.match to accept either a string or a compiled filter expression. 2012-09-14 09:41:01 +12:00
Aldo Cortesi
54cee9db7f Catch a potential exception on connection finalization. 2012-09-14 09:40:13 +12:00
Aldo Cortesi
b7d89f6919 Don't run replace or header hooks on error. 2012-09-02 12:57:49 +12:00
Aldo Cortesi
5630d3f660 Add help entry for H global header shortcut. 2012-09-02 11:53:00 +12:00
Aldo Cortesi
9c009a872e Add U shortcut to add user-agent strings to global Headers editor. 2012-09-02 11:50:17 +12:00
Aldo Cortesi
3fc9af63c1 Fix a crash when re-editing a path prompt after an error. 2012-08-31 13:41:08 +12:00
Aldo Cortesi
26cefc95e8 Make grid editor file reading more robust. 2012-08-31 13:33:18 +12:00
Aldo Cortesi
a33e90f081 Add a shortcut to header editor to add standard User-Agent strings. 2012-08-31 13:28:04 +12:00
Aldo Cortesi
71ae158d7b Display "No Content" instead of a parse error when there is no content. 2012-08-30 12:51:04 +12:00
Aldo Cortesi
a2f9ca1d4d Add application/javascript to ~a filter asset matcher. 2012-08-30 12:42:31 +12:00
Aldo Cortesi
38ddbcc314 Add a snippet to GridEditor help text explaining that we're using escaped strings. 2012-08-25 13:29:05 +12:00
Aldo Cortesi
87463049f1 Add a "R" shortcut to GridEditor, letting the user read unescaped data from file. 2012-08-25 13:25:59 +12:00
Aldo Cortesi
514e19b172 Do away with explicit encodings, and display an error message for invalid values. 2012-08-25 13:13:16 +12:00
Aldo Cortesi
c6d1fe9e59 Use Python-style escaped strings in GridEditor. 2012-08-25 12:54:34 +12:00
Aldo Cortesi
82893ffae2 Add an "r" shortcut in grid editors to read value from file. 2012-08-25 12:21:45 +12:00
Aldo Cortesi
3787f8befb Add a graphical editor for path components (e -> p shortcut from request view screen). 2012-08-19 13:15:54 +12:00
Aldo Cortesi
a77ccc406d Getter and setter for path component on Requests. 2012-08-19 13:03:21 +12:00
Aldo Cortesi
1b7990897e Command-line options for header setting. 2012-08-19 00:14:16 +12:00
Aldo Cortesi
3e96015e61 Add SetHeaders, analogous to ReplaceHooks, with a graphical editor in mitmproxy (H shortcut). 
SetHeaders defines headers that are set on flows, based on a matching pattern.
Existing headers are over-ridden.
 2012-08-18 23:41:04 +12:00
Aldo Cortesi
53e453f72e Use the new ODict get_first convenience function in a bunch of places. 2012-08-18 18:14:30 +12:00
Aldo Cortesi
15e234558d Further content view cleaups. 2012-08-18 17:51:34 +12:00
Aldo Cortesi
b70e91bbd4 Send tracebacks from content viewers to event log. 
Also, 100% test coverage for content viewers.
 2012-08-18 17:42:40 +12:00
Aldo Cortesi
e8553f966f Further simplifcation and testing of contentviews. 2012-08-18 17:29:29 +12:00
Aldo Cortesi
11c63dcb9f Huge cleanup of content viewers. 2012-08-18 17:08:17 +12:00
Sahn Lam
3189d144a5 Optional AMF decoding support 
If PyAMF is installed, enable AMF decoding.
 2012-08-17 18:45:26 -07:00
Maximilian Hils
b0566b9d4c add dummy cert dir 2012-08-17 19:13:56 +02:00
Maximilian Hils
ed389d8f05 use argparse instead of optparse 2012-08-17 19:11:59 +02:00
Aldo Cortesi
bbaa8bdba5 Add an HTML text outline view. 
Uses html2text, from here:

https://github.com/aaronsw/html2text
 2012-08-18 01:37:30 +12:00
Aldo Cortesi
1bfe847a84 Stop server playback after current playback buffer is exhausted. 2012-08-18 00:23:41 +12:00
Aldo Cortesi
20b270ae9a Server replay from current buffer. 2012-08-18 00:13:04 +12:00
Jim Lloyd
0ef18a7cba Adds --dummy-certs option to specify certdir 
If --dummy-certs=CERTSDIR is provided, use CERTSDIR as the location
for generating/finding the dummy certs. And in this case, preserve
the CERTSDIR directory on exit.
 2012-08-06 14:09:35 -07:00
Aldo Cortesi
32ad26f8bf Add a size() method to flow.Request and flow.Response. 2012-08-04 13:18:05 +12:00
Rob Wills
8e68426ad6 Return from Flow View behaviour 
Following focus trumps the re-focusing the "current" (or just-viewed) flow.
 2012-08-02 00:06:34 -07:00
Rob Wills
c985e22196 Press "F" to Follow 
Following of the flow list
- ConsoleState now has a follow_focus property
-- when True it will cause focus to follow added flows
- ConsoleMaster implements toggle_follow_flows()
-- when enabling follow, also jumps to most recent flow
- StatusBar reports follow_focus state
-- adds "following" to option list when true
- Added "F" to FlowList keys
-- implementation requests ConsoleMaster to toggle_follow_flows()
 2012-08-01 23:57:56 -07:00
Aldo Cortesi
87d05a95ff Handle invalid headers. 2012-07-30 12:54:50 +12:00
Aldo Cortesi
bb124e23b2 Prevent mitmproxy from treating logged info as urwid markup. 2012-07-27 10:08:10 +12:00
Aldo Cortesi
47b5fd666d Fix mitmproxy file reading. 2012-07-27 02:13:21 +12:00
Aldo Cortesi
31a092f6b4 Minor refactoring. Make stop_stream also close the associated file descriptor. 2012-07-27 00:19:18 +12:00
Aldo Cortesi
f93a621856 Only log real errors in WSGI apps. 2012-07-24 16:18:22 +12:00
Aldo Cortesi
b4e9e55c34 Be more tolerant of corrupted or truncated flows. 
We load as far as possible. mitmproxy will only terminate if it was not able to
recover any flows. mitmdump will stop loading as soon as an error is
encountered, but not exit with an error.
 2012-07-24 15:15:41 +12:00
Maximilian Hils
c643234c98 fix -r option, read file in binary mode 2012-07-24 03:11:28 +02:00
Chris Neasbitt
525a8f6a16 Fixed a bug causing an AttributeError when request is set to false but response not None in ProxyHandler.handle_request 2012-07-17 13:24:15 -04:00
Aldo Cortesi
e4079aa746 Add an ~a filter expression, matching an asset content type in responses. 
Asset content types are Javascript, images, Flash and CSS. This is useful
because doing a quick "!~a" while auditing an app will filter out the majority
of the static asset cruft, letting you focus on what matters.
 2012-07-14 16:55:21 +12:00
Aldo Cortesi
150814f6a8 Jump back to correct flow when limit is applied. 2012-07-14 15:22:44 +12:00
Aldo Cortesi
a44a76a7da Merge branch 'master' of github.com:cortesi/mitmproxy 2012-07-11 07:17:20 +12:00
Aldo Cortesi
1d09a558a7 Fix a subtle termination condition when there's an error in a WSGI app. 2012-07-11 07:16:06 +12:00
Aldo Cortesi
01b8b0d876 Refine semantics of replay_request method. 2012-07-10 23:29:33 +12:00
Aldo Cortesi
04d9ec8c3c Make WSGI apps work in transparent mode. 2012-07-10 15:53:53 +12:00
Aldo Cortesi
79af9e89c4 Test replay corner cases. Fix discovered bugs. 2012-07-09 11:18:03 +12:00
Aldo Cortesi
aab45078ad Unit test request replay thread. 
This is a small patch, but is the culmination of lots of work: we can now unit
test the deep innards of mitmproxy, with coverage. There's a lot more to come
in this vein.
 2012-07-09 11:03:55 +12:00
Aldo Cortesi
3749d52b66 Make mitmproxy "W" over-write, not append. 
The -w options in mitmdump and mitproxy should do the same thing. Append can
have unexpected consequences if the existing file is not a mitmdump.
 2012-07-09 10:28:01 +12:00
Aldo Cortesi
729fd9301f Use FlowMaster stream from mitmdump. 2012-07-09 10:26:45 +12:00
Aldo Cortesi
bbd9acf551 Use FlowMaster streaming for "W" in mitmproxy. 2012-07-09 10:22:14 +12:00
Aldo Cortesi
572e8a4962 Add streaming to FlowMaster 2012-07-09 10:18:37 +12:00
Aldo Cortesi
097b566e54 Handle new netlib.tcp.NetLibDisconnect exception. 2012-07-08 23:49:44 +12:00
Aldo Cortesi
ceb12438b6 Fix minor error in filter help. 2012-07-06 22:33:09 +12:00
Aldo Cortesi
d02bcade3a Add a domain match filter (~d regex) 2012-07-06 22:21:44 +12:00
Aldo Cortesi
c4426952ad Protect against exceptions thrown by third-party view libraries. 2012-07-06 15:43:33 +12:00
Aldo Cortesi
a7e64a1a03 mitmproxy: "W" shortcut key streams flows to file as responses arrive. 2012-07-06 14:41:10 +12:00
Aldo Cortesi
de294da2a7 Amalgamte saving "this flow" and "all flows" into a single keyboard shortcut. 2012-07-06 14:10:58 +12:00
Aldo Cortesi
40d63c9e7a Fix error when viewing extended details of SSL flow. 2012-07-06 13:33:55 +12:00
Aldo Cortesi
aa708a2d28 Fix error when serializing reverted SSL flows. 2012-07-05 11:52:56 +12:00
Aldo Cortesi
4acc9aca27 Firm up handling of Unicode data 
- Modify GridEditor to know about the destination encoding of data
- Ensure that get_url always returns ASCII
 2012-07-05 11:27:40 +12:00
Aldo Cortesi
837fcc65f5 Make upstream-cert the default. There's now a --no-upstream-cert option to turn it off. 2012-07-03 22:56:25 +12:00
Aldo Cortesi
fe86194cc2 Fix Python coredump (!!) on SNI IDNA decoding. 2012-07-03 22:55:02 +12:00
Aldo Cortesi
9c30e2e86d Correct handing of IDNA encoding of internationalized domain names. 
- Use IDNA encoding for hostnames gleaned by upstream-cert sniffing
- Use IDNA decoding for URL display in mitmproxy and mitmdump.
 2012-07-03 22:27:16 +12:00
Aldo Cortesi
ef986202ee Make server version configurable. 2012-07-03 14:12:52 +12:00
Aldo Cortesi
35d5da9f11 Move palette option getting/setting to mitmproxy. 
Unbreaks mitmdump.
 2012-07-03 13:42:09 +12:00
Aldo Cortesi
c664801d7d Merge pull request #48 from afh/pull/palette-option 
Pull/palette option
 2012-07-01 14:04:43 -07:00
Alexis Hildebrandt
a695cf177d Improve palette colors for headings 2012-07-01 09:03:20 +02:00
Alexis Hildebrandt
e41c84335d Add --palette option to select color palette 2012-07-01 08:48:30 +02:00
Aldo Cortesi
90365e270e Catch and handle SSL connection errors. 2012-07-01 12:10:32 +12:00
Aldo Cortesi
4e9d4e8ddd Tweak upstream SNI. 2012-07-01 11:53:46 +12:00
Aldo Cortesi
9985cf5473 Flush after terminal output. 
Makes behaviour on the shell less mysterious.
 2012-07-01 09:37:03 +12:00
Aldo Cortesi
35fdd16940 Serialize address as part of ClientConnect objects. 
This is a serialization format change!
 2012-07-01 00:16:30 +12:00
Aldo Cortesi
d74a341e5d Beef up logging substantially. 2012-07-01 00:15:03 +12:00
Aldo Cortesi
9fd4c37834 Merge pull request #46 from afh/pull/palettes 
Color palettes
 2012-06-29 21:06:20 -07:00
Aldo Cortesi
f070e4523a Handle invalid data more gracefully. 
Fixes #47
 2012-06-30 15:59:42 +12:00
Aldo Cortesi
16e87a81ac Stub out a transparent resolver for OSX. 2012-06-30 15:42:10 +12:00
Aldo Cortesi
38ebc81590 Add error when -T is passed on an unsupported platform. 2012-06-30 11:24:41 +12:00
Alexis Hildebrandt
0a0a6aae0b Update title palette colors for light backgrounds 2012-06-29 16:58:28 +02:00
Alexis Hildebrandt
3b80e8dd02 Add palettes for Solarized terminal color profiles 2012-06-29 12:45:02 +02:00
Alexis Hildebrandt
e287eac462 Add light palette for light backgrounds 2012-06-29 12:44:57 +02:00
Alexis Hildebrandt
3060ad5575 Replace double quotes with single quotes 2012-06-29 09:25:29 +02:00
Aldo Cortesi
47651b1ff2 Serialization and de-serialization of new cert format. 2012-06-28 14:29:15 +12:00
Aldo Cortesi
243e0efefc Adjust for new get_remote_cert API. 2012-06-28 10:02:14 +12:00
Aldo Cortesi
35ee0c098f Remove certutils from mitmproxy. 2012-06-27 16:43:33 +12:00
Aldo Cortesi
49dedd361c Fix replay. 2012-06-27 16:22:25 +12:00
Aldo Cortesi
dd55a3e0b6 Use SNI-indicated hostname for cert generation when not using upstream certs. 2012-06-27 12:12:11 +12:00
Aldo Cortesi
ceef6ee6be Enable SSL in transparent mode. 2012-06-26 23:51:38 +12:00
Aldo Cortesi
e6cdbefb3b Add transparent mode platform module for Linux. 2012-06-26 20:49:34 +12:00
Aldo Cortesi
ad893ad134 Transparent proxy command-line flag stub. 2012-06-26 20:08:24 +12:00
Aldo Cortesi
8ccfb376f3 Remove -T and -U command-line options. 
They're redundant convenience options, and we need more space.
 2012-06-26 19:57:59 +12:00
Aldo Cortesi
015a74fd14 We no longer store scheme on ServerConnection. 2012-06-26 18:29:12 +12:00
Aldo Cortesi
52d0536d2c Use new TCPClient.convert_to_ssl API. 2012-06-25 15:53:26 +12:00
Aldo Cortesi
e08f91c237 Port to explicit netlib connection API. 2012-06-25 11:37:12 +12:00
Aldo Cortesi
eac3b29d5f Factor read_response out into netlib. 2012-06-24 22:01:11 +12:00
Aldo Cortesi
4db2abc01c read_headers now returns an ODictCaseless object. 2012-06-24 21:49:59 +12:00
Aldo Cortesi
e7c75933e7 read_http_body -> read_http_body_request/response 2012-06-23 15:08:01 +12:00
Aldo Cortesi
874649f134 Adapt for API changes in netlib. 2012-06-23 14:06:34 +12:00
Aldo Cortesi
7cb242c168 Move wsgi to netlib. 2012-06-19 10:42:55 +12:00
Aldo Cortesi
1b1ccab8b7 Extract protocol and tcp server implementations into netlib. 2012-06-19 09:58:50 +12:00
Aldo Cortesi
7b9756f48e Refactor protocol.py to remove dependence on flow and utils. 2012-06-17 10:52:39 +12:00
Aldo Cortesi
aae8a9959c Pull out protocol components into protocol.py 2012-06-16 21:23:32 +12:00
Aldo Cortesi
d5a0099f49 Test suite and refactoring for netlib. 2012-06-16 16:22:51 +12:00
Aldo Cortesi
18a03c063e Simplify netlib and improve API. 2012-06-16 13:53:24 +12:00
Aldo Cortesi
4e53f1ee90 Rename our tcpserver to netlib, expand to include client network functions. 2012-06-16 13:38:10 +12:00
Aldo Cortesi
8ae64337ed Create our own TCP server class. 
We're going to need more control for advanced features and speed, and we can
also ditch some of the idiocies in the SocketServer module.
 2012-06-16 11:40:44 +12:00
Aldo Cortesi
c7952371b7 Fix a problem in ODictCaseless that could cause duplicate headers. 2012-06-15 17:40:08 +12:00
Aldo Cortesi
8ae3270807 Basic transparent mode. 2012-06-15 09:47:04 +12:00
Aldo Cortesi
a9495dc02f Refactor test suite to make room for transparent mode tests. 2012-06-15 09:20:10 +12:00
Aldo Cortesi
176d819559 Move server comms to OpenSSL. 2012-06-14 21:57:55 +12:00
Aldo Cortesi
8dabf88ae5 Remove ability to specify SSL ciphers. 
We can re-introduce this if there's demand - the feature needs a bit more thought.
 2012-06-14 15:29:54 +12:00
Aldo Cortesi
8a9352b3f7 First draft conversion of server to PyOpenSSL. 2012-06-13 18:16:47 +12:00
Aldo Cortesi
d032504b17 Fix an exception when replaying a flow with no response. 2012-06-10 17:10:43 +12:00
Aldo Cortesi
d60fa9918b Localise client connection object manipulation. 
This simplifies the call signature for a bunch of functions.
 2012-06-10 16:49:59 +12:00
Aldo Cortesi
1f659948cd Refactor request processing at mitmproxy's core. 
Gradually cleaning up towards a state machine model.
 2012-06-10 16:02:48 +12:00
Aldo Cortesi
236447c65f Pass server HTTP version back to clients. 2012-06-10 13:29:09 +12:00
Aldo Cortesi
6ba5f0f35b Add HTTP version to response objects. 
Another change in the serialization format.
 2012-06-10 13:27:43 +12:00
Aldo Cortesi
52779d9db9 Refactoring of proxy.py 
- Correctly pass HTTP request version on to upstream servers
- Adjust tests not to hang due to a pathod response with no content-length
 2012-06-10 13:17:18 +12:00
Aldo Cortesi
55ddf853cd Add HTTP version to flow.Request 
This is a serialization format change, that makes us incompatible with previous
versions.
 2012-06-10 10:46:22 +12:00
Aldo Cortesi
a3b47e0cb5 Consolidate HTTP major and minor versions into a single variable. 2012-06-10 10:31:04 +12:00
Aldo Cortesi
8254187bf3 Add proxy.should_connection_close, and strip out unused code. 2012-06-10 10:10:46 +12:00
Aldo Cortesi
0c458e2f1a Refactor ServerConnection API. 2012-06-10 08:13:50 +12:00
Aldo Cortesi
987f443b5d Ignore incorrectnesses in traffic if they don't affect us. 2012-06-09 21:45:22 +12:00
Aldo Cortesi
9130cd63d3 Significant cleanup of proxy internals. 
Dispense with the loose parsing of client requests that we had before. We now
have service modes ("proxy" and "reverse proxy" for now), and we only accept
requests that are appropriate for the mode we're in.
 2012-06-09 21:27:43 +12:00
Aldo Cortesi
05492baf8d Move from requests to human_curl. 
It turns out that _none_ of the Python stdlib or anything that relies on it
supports CONNECT through a proxy. Beggars belief, but there you go.
 2012-06-09 16:17:51 +12:00
Aldo Cortesi
22192d1a46 Nose mopup: docs, no cover pragmas, a few missing path specs. 2012-06-09 13:55:55 +12:00
Aldo Cortesi
b7b357528c Port mitmproxy test suite entirely to nose. 2012-06-09 13:42:43 +12:00
Aldo Cortesi
903038b8de Merge branch 'master' into develop 2012-06-07 08:44:36 +12:00
Maximilian Hils
2a194f98ec add missing parser.add_option_group(group) 2012-06-06 01:26:19 +02:00
Aldo Cortesi
e9109812e1 Split parsing of intial line into separate protocols. 2012-06-03 06:04:57 -07:00
Aldo Cortesi
0a25c2263d Factor out conversion to SSL connection. 2012-06-03 01:54:11 -07:00
Aldo Cortesi
491f9bdcee Add unit tests for console/help.py 2012-06-03 01:11:07 -07:00
Aldo Cortesi
b36e37f9da Fix a traceback that sometimes occurred when an invalid limit was entered. 2012-06-02 18:41:44 -07:00
Aldo Cortesi
ee2950cd19 Fix a crashing bug when replacing text in a flow with unicode bodies. 2012-05-25 18:10:31 -07:00
Aldo Cortesi
a0c63b6108 Fix crash when adding a Response when none existed before. 2012-05-24 16:56:18 -07:00
Paul
5f8855df55 Added a switch to send client certificates to hosts 2012-05-23 23:09:03 +02:00
Aldo Cortesi
08d6da2941 Fix AttributeError when shutting down a writing mitmdump. 2012-05-17 09:47:04 +12:00
Aldo Cortesi
0a90a3eaba Refuse to replay a request with missing content. 2012-05-16 18:24:32 +12:00
Aldo Cortesi
b3901a7652 Add interface to mark content missing, and extend UI to indicate when content is missing. 2012-05-16 18:16:44 +12:00
Aldo Cortesi
0c2d894cea Add the ability to flag content as missing in a request or a response. 
We'll use this in a number of situations. First, we'll soon have response
streaming that directly pipes responses to clients. These will be content-less
from mitmproxy's perspective. Second, we'll be growing new events that fire
after headers are received, but before content is read.
 2012-05-16 15:42:58 +12:00
Aldo Cortesi
12b8a43dbe Bump version number. 2012-05-11 09:52:17 +12:00
Aldo Cortesi
d42fdc4ff6 Include a formatted exception in WSGI error page. 2012-05-07 21:34:01 +12:00
Aldo Cortesi
116fcfcf7a Internal error page for WSGI. 
Also, 100% test coverage.
 2012-04-27 15:56:42 +12:00
Aldo Cortesi
2fe54d17df Don't specify Content-Length on empty content. 
Sometimes, mitmproxy would specify a content-length header value of 0 when
content was empty. Some rare servers (like piratebay.org) would barf on this.
 2012-04-25 14:38:20 +12:00
Aldo Cortesi
efa98d514c Docs and comments for WSGI app example. 2012-04-24 14:58:18 +12:00
Aldo Cortesi
c8d2b2594b Add a WSGI adapter that lets us serve a WSGI app out of mitmproxy. 
This commit adds:
    - A WSGI App adapter for mitmproxy
    - An app registry in the proxy instance that lets us link WSGI apps with
    (hostname, port) combinations.
    - Fixes for a number of bugs discovered while creating this feature.
 2012-04-24 14:52:29 +12:00
Aldo Cortesi
51789228be Fix PKCS12 cert generation. 2012-04-15 12:48:07 +12:00
Aldo Cortesi
2162ce1ae3 Shut mitmdump down gracefully on SIGTERM. 
This is a hack at the moment, but needs must.
 2012-04-15 10:35:19 +12:00
Aldo Cortesi
8c96264304 Serialized data version check. 2012-04-11 10:10:53 +12:00
Aldo Cortesi
24a8dc408c Minor cruft removal. 2012-04-09 11:42:56 +12:00
Aldo Cortesi
d7748cea4f Minor name refactoring, make flow list refresh after limit is applied. 2012-04-09 10:05:13 +12:00
Aldo Cortesi
8fa96d1f3f Add an index counter to status bar. 2012-04-09 09:48:39 +12:00
Aldo Cortesi
5553eb6371 Add some missing help entries. 2012-04-08 20:16:25 +12:00
Aldo Cortesi
4e2d19714c Add an "f" shortcut key to load full body contents. 2012-04-08 19:44:01 +12:00
Aldo Cortesi
618a9c0e2b Merge branch 'master' of ssh.github.com:cortesi/mitmproxy 2012-04-08 15:15:02 +12:00
Aldo Cortesi
c1788c37a1 Wrap on any character for body text display. 2012-04-08 15:00:38 +12:00
Aldo Cortesi
7d90eb65ed Catch all errors when using jsbeautifier. 
Turns out there are some problems that can raise arbitrary exceptions.
 2012-04-08 14:40:59 +12:00
Aldo Cortesi
a4f7728fad XML/HTML pretty view tweaks. 2012-04-07 22:15:31 +12:00
Aldo Cortesi
f1dc3f2ab2 Integrate lxml for pretty-printing HTML and XML. 
Tackling the pretty-printing performance problem head-on, at the cost of a
major dependency.
 2012-04-07 13:47:03 +12:00
Aldo Cortesi
8385e586c0 Work around an Urwid crashing bug. 
https://github.com/Nic0/tyrs/issues/115
 2012-04-05 10:21:56 +12:00
Aldo Cortesi
5fdf710c81 Always use ODictCaseless for headers. 2012-04-04 19:46:39 +12:00
Aldo Cortesi
786e304bb9 Android configuration docs. 2012-04-04 15:58:46 +12:00
Aldo Cortesi
4da8054e21 Docs: replacements, upstream certs. 
Also, move reverse proxy command-line flag to -P.
 2012-04-04 14:17:26 +12:00
Aldo Cortesi
8110a9a3eb Start prepping docs for 0.8 
Also add an interactive upstream-cert option to mitmproxy, and repair help for
R shortcut.
 2012-04-04 11:24:58 +12:00
Aldo Cortesi
79a0334a02 Improve revert model 
- Flows are backed up whenever an interactive, non-script change is made.
- That backup is canonical and never changed - "V" will always revert to it.

This makes more sense than what we had previously....
 2012-04-04 09:47:57 +12:00
Aldo Cortesi
ab0e10e60f Serialize requestcount for ClientConnect objects. 2012-04-03 22:37:24 +12:00
Aldo Cortesi
b9737ed89e Return a datetime object from SSLCert notbefore and notafter properties. 2012-04-03 22:23:07 +12:00
Aldo Cortesi
c6896d7392 Add ClientConnect info to details view. 2012-04-03 11:28:07 +12:00
Aldo Cortesi
61fab03b24 Add a details page, available from a flow view with the 'X' shortcut 
At the moment, this shows the upstream SSL certificate details. More
fine-grained detail that doesn't fit in the flow view itself will be added.
 2012-04-03 11:10:25 +12:00
Aldo Cortesi
f526e5fa12 Minor unit test bump. 2012-04-03 09:52:26 +12:00
Aldo Cortesi
4979a22d3e Add accessor method for SSLCert object on Response. 2012-04-02 17:02:23 +12:00
Aldo Cortesi
ab1d8fa350 Expand SSL cert support 
- Capture the remote SSL certificate
- Expose the remote cert as an attribute on Response
- Expand the certutils.SSLCert interface to expose more cert info
 2012-04-02 16:21:23 +12:00
Aldo Cortesi
bb03255da0 Nicer coloring for HTTP response codes. 
20x: green
30x: blue
40x: orange
50x: light red
other (non-standard or uncommon): dark red
 2012-04-02 14:08:30 +12:00
Aldo Cortesi
c02fdb2463 Refactor proxy.Server to fix a crash when replaying with -n 2012-04-02 13:24:51 +12:00
Aldo Cortesi
f13e2213ea Make it clearer when we fall back to Raw. 2012-04-02 11:22:01 +12:00
Aldo Cortesi
42d06a05c0 Add an option to allow clearing of display mode override. 2012-04-02 11:16:20 +12:00
Aldo Cortesi
7ed1c1c231 More sensible display mode override indicator. 2012-04-02 11:10:55 +12:00
Aldo Cortesi
c6ee813479 Only cache the calculated body of a view. 
This simplifies the code, and loses no time.
 2012-04-02 11:01:38 +12:00
Aldo Cortesi
2df2fe0e4c Move "M" shortcut key to global scope. 
You now don't have to be viewing a flow to change global default view.
 2012-04-02 10:49:57 +12:00
Aldo Cortesi
15cc09f1b8 Start rationalizing content views. 
We now no longer have distinction between "pretty" view and hex/raw. Instead,
we simply a default AUTO view with a global override (M) and a local override
(m).
 2012-04-02 10:30:35 +12:00
Aldo Cortesi
7fef0ecdf5 Make "T" pretty view over-ride persistent when switching between flows. 
We do this by adding a flow settings mechanism to ConsoleState. This is pretty
rough at the moment and should become more sophisticated as needed.
 2012-04-02 09:30:38 +12:00
Aldo Cortesi
35f4a1c424 Tune content viewing to maintain responsiveness: 
- Reduce display cutoff to 20k.
- Make sure that we only indent the visible part of a JS body, not the whole
thing.
 2012-04-01 10:09:25 +12:00
Aldo Cortesi
585bf9423f Fix GridEditor external edit functionality. 2012-03-30 19:51:40 +13:00
Aldo Cortesi
6dc945571d When paging through flows in flowview, indicate when there are no more flows. 2012-03-28 10:21:48 +13:00
Aldo Cortesi
a6df72cfc8 Generate better-formed CAs. 
If you're having trouble with your certs in Firefox, you may want to delete
your .mitmproxy directory to re-generate the CA.
 2012-03-27 17:01:18 +13:00
Aldo Cortesi
e9ac4bef20 Add a variant of cleanBin that escapes newlines and tabs. 
Use this to fix the hex display option.
 2012-03-27 11:25:50 +13:00
Aldo Cortesi
a2fddb4404 Add image pretty viewer to forcing options. 2012-03-26 11:48:35 +13:00
Aldo Cortesi
d187965233 Don't auto-view request if there's no response. 
This is confusing when flicking through many flows.
 2012-03-26 11:28:52 +13:00
Aldo Cortesi
a050eeef05 Add a pretty-viewer for images. 
This shows basic image information like dimensions, plus extracted EXIF tags
and other metadata.
 2012-03-26 11:26:02 +13:00
Aldo Cortesi
b6725ee802 Add Javascript to manual pretty type over-rides. 2012-03-25 11:31:36 +13:00
Aldo Cortesi
a979e1ad50 Re-enable caching for content view calculation. 2012-03-25 11:18:33 +13:00
Aldo Cortesi
2240d2a6a5 Pretty view now indents Javascript. 
Thanks to the JSBeautifier project, which is now included in the contrib directory.
 2012-03-25 10:56:45 +13:00
Aldo Cortesi
74c51df580 Re-enable simple multipart form parsing and preview. 2012-03-25 10:10:48 +13:00
Aldo Cortesi
62e51018d0 Refactor pretty view mechanism. 
Also start adding unit tests for this subsystem.
 2012-03-24 14:20:24 +13:00
Aldo Cortesi
0d05068f91 Factor out content view apparatus into contentview.py 2012-03-24 11:21:58 +13:00
Aldo Cortesi
ed74ed24a0 Add error indications to GridEditor. 2012-03-23 13:28:33 +13:00
Aldo Cortesi
45ab22f0d9 Add modal help to GridEditor. 
Also includes some minor refactoring and documentation stubs.
 2012-03-22 14:57:57 +13:00
Aldo Cortesi
1441fade90 More work on pretty view forcing. 
- Make this setting apply only to the currently viewed flow. I think it's more
likely that this is the behaviour the user will want, rather than globally
setting the pretty type.
- Update help.
 2012-03-20 11:11:53 +13:00
Aldo Cortesi
2153835545 Refactor pretty view forcing somewhat. 
- Use a lookup table of content types -> view modes.
- Add a urlencoded forcing. Remove "html" - at the moment it's the same as
"xmlish".
- Display type when forced.
 2012-03-20 10:58:43 +13:00
Aldo Cortesi
2739cb4861 Add a simple parser for content type specifications. 2012-03-20 10:31:07 +13:00
Ulrich Petri
bc3ba4c993 Allow a flow view's pretty-printed body type auto-detection to be overridden by the user 2012-03-19 20:35:20 +01:00
Aldo Cortesi
50630d62fd Grid editor refinements. 
- Correct column width calculation used in keystrokes. Arrow keys should now
work as expected in multi-line fields.
- Enable multi-line editing throughout. You now need to tab or esc to exit edit
mode.
 2012-03-19 10:57:27 +13:00
Aldo Cortesi
0de97ad9e0 Add column headers to GridEditor. 2012-03-19 10:42:30 +13:00
Aldo Cortesi
65e88f49d4 Specialize GridEditor into a number of subclasses. 2012-03-19 10:12:06 +13:00
Aldo Cortesi
5690e7c399 Generalize GridEditor to N columns. 
Start adding a replacement rule editor.
 2012-03-18 14:39:21 +13:00
Aldo Cortesi
18d0e840b5 KVEditor -> GridEditor 2012-03-18 09:48:27 +13:00
Aldo Cortesi
552146d015 Move mitmproxy reverse proxy shortcut to "P". 
Also don't expose help from the help screen.
 2012-03-18 09:22:43 +13:00
Aldo Cortesi
ac6987c54a Fix unit tests. 2012-03-17 17:29:25 +13:00
Aldo Cortesi
76175672ad Add specification of replacement patterns on the command line. 2012-03-17 17:20:34 +13:00
Aldo Cortesi
c8ae1e85b3 Hooks -> ReplaceHooks 
It makes more sense to specialize this, which will let me build a nicer
interface for replacement hooks in mitmproxy.
 2012-03-17 11:31:05 +13:00
Aldo Cortesi
08f410cacc Add a hooks mechanism, based on filter expressions. 2012-03-16 17:13:11 +13:00
Aldo Cortesi
d138af7217 replace() methods now decode and re-encode contents before substitution. 2012-03-16 11:24:18 +13:00
Aldo Cortesi
d51b8cab0c Add a decoded context manager. 
This simplifies a common chore when modifying traffic - decoding the object,
modifying it, then re-encoding it with the same encoding afterwards. You can
now simply say:

with flow.decoded(request):
    request.content = "bar"
 2012-03-16 11:12:56 +13:00
Aldo Cortesi
8d662e6636 Set a "unique" serial number for each generated cert. 2012-03-14 11:20:25 +13:00
Aldo Cortesi
fa6305ee98 Cleanliness fixes. 
- Remove unused code during previous commit.
- Code coverage fixes.
 2012-03-12 11:25:50 +13:00
Aldo Cortesi
22d4559a7a Use PyOpenSSL for certificate generation. 
We no longer call external OpenSSL commands at all.
 2012-03-11 14:34:17 +13:00
Aldo Cortesi
4e13ab1d05 Replace CA generation with PyOpenSSL version. 
We no longer call an external command.
 2012-03-11 13:33:22 +13:00
Aldo Cortesi
d57a1d6035 Merge remote-tracking branch 'meeee/master' 2012-03-10 13:48:13 +13:00
Aldo Cortesi
3f2d1381d0 Repair unit tests after nopop introduction. 2012-03-10 13:42:10 +13:00
Aldo Cortesi
d3aad7a185 Merge remote-tracking branch 'taiste/server-replay-pop' 2012-03-10 13:36:50 +13:00
Michael Frister
23f7214fc3 Fix SSL requests with Transfer-Encoding: chunked 
Add size parameter to FileLike.readline, used by read_chunked.
 2012-03-08 23:10:21 +01:00
Michael Frister
e67dbf6123 Handle Transfer-Encoding header values case insensitive 
According to HTTP/1.1 RFC 2616 Section 3.6.
 2012-03-08 23:09:19 +01:00
Valtteri Virtanen
93565392cd Adds space between sentences on no-pop's help 2012-03-05 11:56:03 +02:00
Valtteri Virtanen
ed56d67cea Adds no-pop option to server-side replay 2012-03-05 11:05:11 +02:00
Aldo Cortesi
e1356dd2b6 Create an SSL certificate class. 2012-03-05 10:22:47 +13:00
András Veres-Szentkirályi
15ad7704d2 Removed imports left unused after Py{OpenSSL,ASN1} 
Commits 533f61f67a and
8b841bc9e3 left some imports unused while
swithing to PyOpenSSL and PyASN1 -- this commit removes these imports.
 2012-03-01 16:20:34 +01:00
Aldo Cortesi
533f61f67a Use PyOpenSSL and PyASN1 for certificate parsing. 
Yes, these are two more major dependencies for mitmproxy, but if we're going to
do all the cool things I want to do with SSL certs, there is no other way.
 2012-03-01 21:08:44 +13:00
Aldo Cortesi
8b841bc9e3 Factor out cert operations in to certutils.py. 2012-02-29 13:20:53 +13:00
Aldo Cortesi
0bed5fae27 Rationalise upstream cert flag and variable names. 2012-02-28 11:37:48 +13:00
Aldo Cortesi
764724748b Fix cert generation harder. 2012-02-27 15:59:29 +13:00
Aldo Cortesi
2c73e8f816 Fix problems with SANs and certificate generation. 2012-02-27 15:36:19 +13:00
Aldo Cortesi
2ba8296843 Better certificate parsing. 2012-02-27 15:21:05 +13:00
Aldo Cortesi
00942c1431 Add upstream certificate lookup. 
This initiates a connection to the server to obtain certificate information to
generate interception certificates. At the moment, the information used is the
Common Name, and the list of Subject Alternative Names.
 2012-02-27 15:05:45 +13:00
Aldo Cortesi
4a2964985c Introduce a cache for flow list entries. 
This gives a big boost to scroll performance for the flow list.
 2012-02-27 10:00:44 +13:00
Aldo Cortesi
bd1d699040 Fix mitmproxy crash when passed -n flag. 2012-02-26 23:23:54 +13:00
Aldo Cortesi
3afa2c38fb Merge remote-tracking branch 'remotes/runeh/master' into runeh 2012-02-25 13:02:12 +13:00
Rune Halvorsen
bbfdc7b7de Use shlex to parse EDITOR. 2012-02-25 00:43:00 +01:00
Aldo Cortesi
986a41d180 Unit test++. 2012-02-25 12:19:54 +13:00
Aldo Cortesi
bcda65e453 Add mitmproxy version to status bar on Help screen. 
Suggested by Jim Cheetham <jim.cheetham@otago.ac.nz>
 2012-02-24 14:11:51 +13:00
Aldo Cortesi
5810e7c0df Make return arrow match return code color. 
Suggested by Jim Cheetham <jim.cheetham@otago.ac.nz>
 2012-02-24 14:01:17 +13:00
Aldo Cortesi
25fa596cd6 Fix detection of URL-encoded forms. 
Thanks to Paul Capestany <capestany@gmail.com> for reporting this.
 2012-02-24 13:03:24 +13:00
Aldo Cortesi
ddc9155c24 Make "~q" filter work more intuitively. 
It now matches any flow that has no response.
 2012-02-23 17:06:09 +13:00
Aldo Cortesi
2df9c52c09 Refactor filter matching. 2012-02-23 17:03:58 +13:00