Grasscutters/mitmproxy
3
0
Fork 0
mirror of https://github.com/Grasscutters/mitmproxy.git synced 2024-11-23 16:17:49 +00:00
Code Issues Projects Releases Wiki Activity
1,148 Commits 1 Branch 67 Tags 50 MiB
9b17d272a3
Commit Graph

382 Commits

Author SHA1 Message Date
Aldo Cortesi
0257815141 Significantly simplify server connection handling, and test. 2013-02-24 22:24:21 +13:00
Aldo Cortesi
705559d65e Refactor to prepare for SNI fixes. 2013-02-24 17:35:24 +13:00
Aldo Cortesi
d0639e8925 Handle server disconnects better. 
Server connections can be closed for legitimate reasons, like timeouts. If
we've already pumped data over a server connection, we reconnect on error. If
not, we treat it as a legitimate error and pass it on to the client.

Fixes #85
 2013-02-24 14:04:56 +13:00
Aldo Cortesi
64285140f9 Test a difficult-to-trigger IOError, fix cert generation in test suite. 2013-02-24 11:34:01 +13:00
Aldo Cortesi
51de9f9fdf Test client connection close conditions. 2013-02-24 10:51:14 +13:00
Aldo Cortesi
05e4d4468e Test request and response kill functionality. 2013-02-23 21:59:25 +13:00
Aldo Cortesi
269780c577 Unit test dummy response functions. 2013-02-23 16:34:59 +13:00
Aldo Cortesi
aaf892e3af Significantly refactor the master/slave message passing interface. 2013-02-17 12:42:48 +13:00
Aldo Cortesi
1ccb2c5dea Test WSGI app calling. 
- Factor out test servers into a separate file
- Adjust docs to note new Flask dependency
 2013-02-16 16:46:16 +13:00
Eric Entzel
6bcf29c0ed Keep blank URL parameters 
TODO: This should probably be configurable
 2013-02-11 13:22:25 +11:00
Aldo Cortesi
782bbee8c0 Unit tests for ServerConnectionPool 2013-01-29 11:35:57 +13:00
Aldo Cortesi
53792a5a28 Beef up unit tests for HAR utility functions - flow.py coverage now 100%. 2013-01-29 10:41:45 +13:00
Aldo Cortesi
6f157d936f Merge pull request #99 from rouli/master 
Adding helper functions to make HAR export easier
 2013-01-28 13:28:49 -08:00
Rouli
330fbfe8cc adding helper functions to make HAR export easier 2013-01-28 17:37:25 +02:00
Aldo Cortesi
57f01ffb07 Test suite, remove extraneous code. 2013-01-28 21:59:03 +13:00
Aldo Cortesi
25cb9471f0 Add tests for client certificate support. 2013-01-20 22:39:28 +13:00
Aldo Cortesi
6600c589ab Rudimentary testing for client certs. 2013-01-18 17:08:30 +13:00
Aldo Cortesi
0f406e9daa Speed up the test suite by reducing time sleeping in tests. 2013-01-18 14:52:19 +13:00
Aldo Cortesi
7a79eeb143 Merge branch 'master' of ssh.github.com:cortesi/mitmproxy 
Conflicts:
	test/test_server.py
 2013-01-18 14:50:31 +13:00
Rouli
6212b69fb4 fixes due to merge with master 2013-01-17 17:36:18 +02:00
Rouli
446f9f0a0f Merge remote-tracking branch 'upstream/master' 2013-01-17 17:33:29 +02:00
Rouli
20fa6a3083 changing requests and responses to have two timestamps, one marking their initiation, and the other their complete 2013-01-17 17:32:56 +02:00
Aldo Cortesi
d0ee4d60d0 Unit tests and minor code refactoring for ServerConnection. 2013-01-05 19:44:12 -08:00
Aldo Cortesi
891c441a6d Use new netlib certificate store implementation. 2013-01-06 01:16:08 +13:00
Aldo Cortesi
9cfc785cd3 Unit test love - 100% for flow.py, dump.py 2013-01-05 21:56:33 +13:00
Aldo Cortesi
46ab6ed491 Minor cleanups of proxy request handling. 2013-01-04 14:19:32 +13:00
Aldo Cortesi
f5e49ef598 First draft of "How mitmproxy works", a complete guide to the mechanics of the proxy process 2013-01-03 17:26:59 +13:00
Aldo Cortesi
e2dc7ba09d First draft of OSX transparent proxy mode. 2013-01-01 11:13:56 +13:00
Aldo Cortesi
5347cb9c26 More work on proxy auth 
- Strip auth header if auth succeeds, so it's not passed upstream
- Actually use realm specification to BasicProxyAuth, and make it mandatory
- Cleanups and unit tests
 2012-12-31 10:56:44 +13:00
Aldo Cortesi
3b84111493 Test and robustify BasicProxyAuth.parse_auth_value 
- This is partly in preparation for moving the implementation to netlib
- Also add an unparse_auth_value for testing (and use in pathod once the move is done)
 2012-12-31 10:34:25 +13:00
Aldo Cortesi
018c229ae4 Start solidifying proxy authentication 
- Add a unit test file
- Remove some extraneous methods
- Change the auth API to make the authenticate method take a header object.
 2012-12-31 09:15:56 +13:00
Aldo Cortesi
8c976ac7f0 Substantially rewrite AMF decoding. 
This is tricky, but we should now handle a lot more corner-cases.
 2012-11-26 13:25:07 +13:00
Aldo Cortesi
0d59fd7e01 Move cleanBin and hexdump into netutils. 2012-09-24 11:21:12 +12:00
Aldo Cortesi
d115b5ae70 Expand Flow.match to accept either a string or a compiled filter expression. 2012-09-14 09:41:01 +12:00
Aldo Cortesi
a77ccc406d Getter and setter for path component on Requests. 2012-08-19 13:03:21 +12:00
Aldo Cortesi
60659a89c3 Little bit of love for the unit tests. 2012-08-19 00:22:42 +12:00
Aldo Cortesi
1b7990897e Command-line options for header setting. 2012-08-19 00:14:16 +12:00
Aldo Cortesi
3e96015e61 Add SetHeaders, analogous to ReplaceHooks, with a graphical editor in mitmproxy (H shortcut). 
SetHeaders defines headers that are set on flows, based on a matching pattern.
Existing headers are over-ridden.
 2012-08-18 23:41:04 +12:00
Aldo Cortesi
b70e91bbd4 Send tracebacks from content viewers to event log. 
Also, 100% test coverage for content viewers.
 2012-08-18 17:42:40 +12:00
Aldo Cortesi
e8553f966f Further simplifcation and testing of contentviews. 2012-08-18 17:29:29 +12:00
Aldo Cortesi
11c63dcb9f Huge cleanup of content viewers. 2012-08-18 17:08:17 +12:00
Sahn Lam
3189d144a5 Optional AMF decoding support 
If PyAMF is installed, enable AMF decoding.
 2012-08-17 18:45:26 -07:00
Aldo Cortesi
a66d018363 Fix unit tests after argparse conversion. 2012-08-18 10:27:31 +12:00
Aldo Cortesi
32ad26f8bf Add a size() method to flow.Request and flow.Response. 2012-08-04 13:18:05 +12:00
Aldo Cortesi
84bffad3fc Fix flow read unit test to accomodate more tolerant dumpfile parsing. 2012-07-24 15:30:52 +12:00
András Veres-Szentkirályi
1a26f8215d removed assigned but unread variables 2012-07-15 22:42:59 +02:00
András Veres-Szentkirályi
da496669c2 removed unused imports 2012-07-15 22:42:45 +02:00
Aldo Cortesi
e4079aa746 Add an ~a filter expression, matching an asset content type in responses. 
Asset content types are Javascript, images, Flash and CSS. This is useful
because doing a quick "!~a" while auditing an app will filter out the majority
of the static asset cruft, letting you focus on what matters.
 2012-07-14 16:55:21 +12:00
Aldo Cortesi
01b8b0d876 Refine semantics of replay_request method. 2012-07-10 23:29:33 +12:00
Aldo Cortesi
04d9ec8c3c Make WSGI apps work in transparent mode. 2012-07-10 15:53:53 +12:00
Aldo Cortesi
79af9e89c4 Test replay corner cases. Fix discovered bugs. 2012-07-09 11:18:03 +12:00
Aldo Cortesi
aab45078ad Unit test request replay thread. 
This is a small patch, but is the culmination of lots of work: we can now unit
test the deep innards of mitmproxy, with coverage. There's a lot more to come
in this vein.
 2012-07-09 11:03:55 +12:00
Aldo Cortesi
e49c920d16 Refator server tests to use flow.FlowMaster and flow.State 2012-07-09 10:58:28 +12:00
Aldo Cortesi
572e8a4962 Add streaming to FlowMaster 2012-07-09 10:18:37 +12:00
Aldo Cortesi
4b6fdc92dc Remove ODict tests. ODict is now in netlib. 2012-07-09 09:54:15 +12:00
Aldo Cortesi
d02bcade3a Add a domain match filter (~d regex) 2012-07-06 22:21:44 +12:00
Aldo Cortesi
aa708a2d28 Fix error when serializing reverted SSL flows. 2012-07-05 11:52:56 +12:00
Aldo Cortesi
90365e270e Catch and handle SSL connection errors. 2012-07-01 12:10:32 +12:00
Aldo Cortesi
d74a341e5d Beef up logging substantially. 2012-07-01 00:15:03 +12:00
Aldo Cortesi
f070e4523a Handle invalid data more gracefully. 
Fixes #47
 2012-06-30 15:59:42 +12:00
Aldo Cortesi
47651b1ff2 Serialization and de-serialization of new cert format. 2012-06-28 14:29:15 +12:00
Aldo Cortesi
35ee0c098f Remove certutils from mitmproxy. 2012-06-27 16:43:33 +12:00
Aldo Cortesi
8ccfb376f3 Remove -T and -U command-line options. 
They're redundant convenience options, and we need more space.
 2012-06-26 19:57:59 +12:00
Aldo Cortesi
874649f134 Adapt for API changes in netlib. 2012-06-23 14:06:34 +12:00
Aldo Cortesi
7cb242c168 Move wsgi to netlib. 2012-06-19 10:42:55 +12:00
Aldo Cortesi
1b1ccab8b7 Extract protocol and tcp server implementations into netlib. 2012-06-19 09:58:50 +12:00
Aldo Cortesi
7b9756f48e Refactor protocol.py to remove dependence on flow and utils. 2012-06-17 10:52:39 +12:00
Aldo Cortesi
aae8a9959c Pull out protocol components into protocol.py 2012-06-16 21:23:32 +12:00
Aldo Cortesi
d5a0099f49 Test suite and refactoring for netlib. 2012-06-16 16:22:51 +12:00
Aldo Cortesi
4e53f1ee90 Rename our tcpserver to netlib, expand to include client network functions. 2012-06-16 13:38:10 +12:00
Aldo Cortesi
c7952371b7 Fix a problem in ODictCaseless that could cause duplicate headers. 2012-06-15 17:40:08 +12:00
Aldo Cortesi
8ae3270807 Basic transparent mode. 2012-06-15 09:47:04 +12:00
Aldo Cortesi
a9495dc02f Refactor test suite to make room for transparent mode tests. 2012-06-15 09:20:10 +12:00
Aldo Cortesi
8a9352b3f7 First draft conversion of server to PyOpenSSL. 2012-06-13 18:16:47 +12:00
Aldo Cortesi
d032504b17 Fix an exception when replaying a flow with no response. 2012-06-10 17:10:43 +12:00
Aldo Cortesi
d60fa9918b Localise client connection object manipulation. 
This simplifies the call signature for a bunch of functions.
 2012-06-10 16:49:59 +12:00
Aldo Cortesi
6ba5f0f35b Add HTTP version to response objects. 
Another change in the serialization format.
 2012-06-10 13:27:43 +12:00
Aldo Cortesi
52779d9db9 Refactoring of proxy.py 
- Correctly pass HTTP request version on to upstream servers
- Adjust tests not to hang due to a pathod response with no content-length
 2012-06-10 13:17:18 +12:00
Aldo Cortesi
55ddf853cd Add HTTP version to flow.Request 
This is a serialization format change, that makes us incompatible with previous
versions.
 2012-06-10 10:46:22 +12:00
Aldo Cortesi
a3b47e0cb5 Consolidate HTTP major and minor versions into a single variable. 2012-06-10 10:31:04 +12:00
Aldo Cortesi
8254187bf3 Add proxy.should_connection_close, and strip out unused code. 2012-06-10 10:10:46 +12:00
Aldo Cortesi
18c1b44475 Reverse proxy testing. 2012-06-09 20:41:28 +12:00
Aldo Cortesi
05492baf8d Move from requests to human_curl. 
It turns out that _none_ of the Python stdlib or anything that relies on it
supports CONNECT through a proxy. Beggars belief, but there you go.
 2012-06-09 16:17:51 +12:00
Aldo Cortesi
22192d1a46 Nose mopup: docs, no cover pragmas, a few missing path specs. 2012-06-09 13:55:55 +12:00
Aldo Cortesi
b7b357528c Port mitmproxy test suite entirely to nose. 2012-06-09 13:42:43 +12:00
Aldo Cortesi
a63240a848 Move pathod service testing truss to nose. 2012-06-09 12:13:01 +12:00
Aldo Cortesi
e78b48ab20 Start conversion to nose. 
RIP pry.
 2012-06-09 10:57:00 +12:00
Aldo Cortesi
7a312546f3 Shift mitmproxy test suite over to pathod. 
This opens a whole brave new world of testing for mitmproxy.
 2012-06-08 10:00:16 +12:00
Aldo Cortesi
e9109812e1 Split parsing of intial line into separate protocols. 2012-06-03 06:04:57 -07:00
Aldo Cortesi
491f9bdcee Add unit tests for console/help.py 2012-06-03 01:11:07 -07:00
Aldo Cortesi
ee2950cd19 Fix a crashing bug when replacing text in a flow with unicode bodies. 2012-05-25 18:10:31 -07:00
Aldo Cortesi
0a90a3eaba Refuse to replay a request with missing content. 2012-05-16 18:24:32 +12:00
Aldo Cortesi
0c2d894cea Add the ability to flag content as missing in a request or a response. 
We'll use this in a number of situations. First, we'll soon have response
streaming that directly pipes responses to clients. These will be content-less
from mitmproxy's perspective. Second, we'll be growing new events that fire
after headers are received, but before content is read.
 2012-05-16 15:42:58 +12:00
Aldo Cortesi
116fcfcf7a Internal error page for WSGI. 
Also, 100% test coverage.
 2012-04-27 15:56:42 +12:00
Aldo Cortesi
c8d2b2594b Add a WSGI adapter that lets us serve a WSGI app out of mitmproxy. 
This commit adds:
    - A WSGI App adapter for mitmproxy
    - An app registry in the proxy instance that lets us link WSGI apps with
    (hostname, port) combinations.
    - Fixes for a number of bugs discovered while creating this feature.
 2012-04-24 14:52:29 +12:00
Aldo Cortesi
8c96264304 Serialized data version check. 2012-04-11 10:10:53 +12:00
Aldo Cortesi
4e2d19714c Add an "f" shortcut key to load full body contents. 2012-04-08 19:44:01 +12:00
Aldo Cortesi
a4f7728fad XML/HTML pretty view tweaks. 2012-04-07 22:15:31 +12:00
Aldo Cortesi
f1dc3f2ab2 Integrate lxml for pretty-printing HTML and XML. 
Tackling the pretty-printing performance problem head-on, at the cost of a
major dependency.
 2012-04-07 13:47:03 +12:00
Aldo Cortesi
ab0e10e60f Serialize requestcount for ClientConnect objects. 2012-04-03 22:37:24 +12:00
Aldo Cortesi
61fab03b24 Add a details page, available from a flow view with the 'X' shortcut 
At the moment, this shows the upstream SSL certificate details. More
fine-grained detail that doesn't fit in the flow view itself will be added.
 2012-04-03 11:10:25 +12:00
Aldo Cortesi
f526e5fa12 Minor unit test bump. 2012-04-03 09:52:26 +12:00
Aldo Cortesi
4979a22d3e Add accessor method for SSLCert object on Response. 2012-04-02 17:02:23 +12:00
Aldo Cortesi
ab1d8fa350 Expand SSL cert support 
- Capture the remote SSL certificate
- Expose the remote cert as an attribute on Response
- Expand the certutils.SSLCert interface to expose more cert info
 2012-04-02 16:21:23 +12:00
Aldo Cortesi
c02fdb2463 Refactor proxy.Server to fix a crash when replaying with -n 2012-04-02 13:24:51 +12:00
Aldo Cortesi
15cc09f1b8 Start rationalizing content views. 
We now no longer have distinction between "pretty" view and hex/raw. Instead,
we simply a default AUTO view with a global override (M) and a local override
(m).
 2012-04-02 10:30:35 +12:00
Aldo Cortesi
7fef0ecdf5 Make "T" pretty view over-ride persistent when switching between flows. 
We do this by adding a flow settings mechanism to ConsoleState. This is pretty
rough at the moment and should become more sophisticated as needed.
 2012-04-02 09:30:38 +12:00
Aldo Cortesi
e9ac4bef20 Add a variant of cleanBin that escapes newlines and tabs. 
Use this to fix the hex display option.
 2012-03-27 11:25:50 +13:00
Aldo Cortesi
a050eeef05 Add a pretty-viewer for images. 
This shows basic image information like dimensions, plus extracted EXIF tags
and other metadata.
 2012-03-26 11:26:02 +13:00
Aldo Cortesi
2240d2a6a5 Pretty view now indents Javascript. 
Thanks to the JSBeautifier project, which is now included in the contrib directory.
 2012-03-25 10:56:45 +13:00
Aldo Cortesi
74c51df580 Re-enable simple multipart form parsing and preview. 2012-03-25 10:10:48 +13:00
Aldo Cortesi
62e51018d0 Refactor pretty view mechanism. 
Also start adding unit tests for this subsystem.
 2012-03-24 14:20:24 +13:00
Aldo Cortesi
ed74ed24a0 Add error indications to GridEditor. 2012-03-23 13:28:33 +13:00
Aldo Cortesi
2739cb4861 Add a simple parser for content type specifications. 2012-03-20 10:31:07 +13:00
Aldo Cortesi
5690e7c399 Generalize GridEditor to N columns. 
Start adding a replacement rule editor.
 2012-03-18 14:39:21 +13:00
Aldo Cortesi
76175672ad Add specification of replacement patterns on the command line. 2012-03-17 17:20:34 +13:00
Aldo Cortesi
c8ae1e85b3 Hooks -> ReplaceHooks 
It makes more sense to specialize this, which will let me build a nicer
interface for replacement hooks in mitmproxy.
 2012-03-17 11:31:05 +13:00
Aldo Cortesi
08f410cacc Add a hooks mechanism, based on filter expressions. 2012-03-16 17:13:11 +13:00
Aldo Cortesi
d138af7217 replace() methods now decode and re-encode contents before substitution. 2012-03-16 11:24:18 +13:00
Aldo Cortesi
d51b8cab0c Add a decoded context manager. 
This simplifies a common chore when modifying traffic - decoding the object,
modifying it, then re-encoding it with the same encoding afterwards. You can
now simply say:

with flow.decoded(request):
    request.content = "bar"
 2012-03-16 11:12:56 +13:00
Aldo Cortesi
fa6305ee98 Cleanliness fixes. 
- Remove unused code during previous commit.
- Code coverage fixes.
 2012-03-12 11:25:50 +13:00
Aldo Cortesi
d3aad7a185 Merge remote-tracking branch 'taiste/server-replay-pop' 2012-03-10 13:36:50 +13:00
Valtteri Virtanen
041eafba73 Added tests for ServerPlaybackState with nopop 2012-03-05 13:57:57 +02:00
Valtteri Virtanen
5b5b79f5c4 Fixed old tests 2012-03-05 13:40:18 +02:00
Aldo Cortesi
e1356dd2b6 Create an SSL certificate class. 2012-03-05 10:22:47 +13:00
András Veres-Szentkirályi
15ad7704d2 Removed imports left unused after Py{OpenSSL,ASN1} 
Commits 533f61f67a and
8b841bc9e3 left some imports unused while
swithing to PyOpenSSL and PyASN1 -- this commit removes these imports.
 2012-03-01 16:20:34 +01:00
Aldo Cortesi
533f61f67a Use PyOpenSSL and PyASN1 for certificate parsing. 
Yes, these are two more major dependencies for mitmproxy, but if we're going to
do all the cool things I want to do with SSL certs, there is no other way.
 2012-03-01 21:08:44 +13:00
Aldo Cortesi
8b841bc9e3 Factor out cert operations in to certutils.py. 2012-02-29 13:20:53 +13:00
Aldo Cortesi
688faa9baa Repair unit tests. 2012-02-27 20:34:47 +13:00
Aldo Cortesi
764724748b Fix cert generation harder. 2012-02-27 15:59:29 +13:00
Aldo Cortesi
2ba8296843 Better certificate parsing. 2012-02-27 15:21:05 +13:00
Aldo Cortesi
00942c1431 Add upstream certificate lookup. 
This initiates a connection to the server to obtain certificate information to
generate interception certificates. At the moment, the information used is the
Common Name, and the list of Subject Alternative Names.
 2012-02-27 15:05:45 +13:00
Aldo Cortesi
986a41d180 Unit test++. 2012-02-25 12:19:54 +13:00
Aldo Cortesi
25fa596cd6 Fix detection of URL-encoded forms. 
Thanks to Paul Capestany <capestany@gmail.com> for reporting this.
 2012-02-24 13:03:24 +13:00
Aldo Cortesi
ddc9155c24 Make "~q" filter work more intuitively. 
It now matches any flow that has no response.
 2012-02-23 17:06:09 +13:00
Aldo Cortesi
2df9c52c09 Refactor filter matching. 2012-02-23 17:03:58 +13:00
Aldo Cortesi
dbd75e02f7 Create ODictCaseless for headers, use vanilla ODict for everything else. 2012-02-20 11:29:36 +13:00
Aldo Cortesi
18029df99c Use ODict for request.get_form_urlencoded and set_form_urlencoded 2012-02-20 11:13:35 +13:00
Aldo Cortesi
b0f77dfefd Unit test import cleanups. 2012-02-20 11:04:07 +13:00
Aldo Cortesi
fa11b7c9be Use ODict for Request.get_query and Request.set_query 2012-02-20 10:44:47 +13:00
Aldo Cortesi
2616f490fe Rename Headers class to ODict 
ODict is an ordered dictionary class that will be useful in many other parts of
our API.
 2012-02-20 10:39:00 +13:00
Aldo Cortesi
d32d6bc5e3 Add "p" key binding to connection list view to copy a flow. 2012-02-19 00:17:47 +13:00
Aldo Cortesi
8ddc3b4ef2 Add API for duplicating flows. 2012-02-18 23:56:40 +13:00
Aldo Cortesi
7aa79b89e8 Firm up what we consider to be a valid proxy spec. 2012-02-18 16:29:02 +13:00
Aldo Cortesi
6ad8b1a15d Firm up reverse proxy specification. 
- Extract proxy spec parsing and unparsing functions.
- Add a status indicator in mitmproxy.
- Add the "R" keybinding for changing the reverse proxy from within mitmproxy.
 2012-02-18 16:27:09 +13:00
Aldo Cortesi
a7df6e1503 Refactor reverse proxying 
- Retain the specification from the Host header as a Request's description.
- Expand upstream proxy specifications to include the scheme. We now say https://hostname:port
- Move the "R" revert keybinding to "v" to make room for a reverse proxy
binding that matches the command-line flag.
 2012-02-18 14:45:22 +13:00
Aldo Cortesi
acdc2d00b4 Repair unit tests. 2012-02-18 12:27:59 +13:00
Aldo Cortesi
da1ccfddeb 100% test coverage for flow.py 2012-02-10 15:55:58 +13:00
Aldo Cortesi
1ad7e91527 Make filter matching act more sensibly. 2012-02-10 15:31:45 +13:00
Aldo Cortesi
5f785e26b9 Add filter for detecting flows with errors. 
Also, remove dependency on weird _is_response method.
 2012-02-10 15:22:26 +13:00