Grasscutters/mitmproxy
3
0
Fork 0
mirror of https://github.com/Grasscutters/mitmproxy.git synced 2024-11-23 08:11:00 +00:00
Code Issues Projects Releases Wiki Activity
590 Commits 1 Branch 67 Tags 50 MiB
8254187bf3
Commit Graph

202 Commits

Author SHA1 Message Date
Aldo Cortesi
2ba8296843 Better certificate parsing. 2012-02-27 15:21:05 +13:00
Aldo Cortesi
00942c1431 Add upstream certificate lookup. 
This initiates a connection to the server to obtain certificate information to
generate interception certificates. At the moment, the information used is the
Common Name, and the list of Subject Alternative Names.
 2012-02-27 15:05:45 +13:00
Aldo Cortesi
986a41d180 Unit test++. 2012-02-25 12:19:54 +13:00
Aldo Cortesi
25fa596cd6 Fix detection of URL-encoded forms. 
Thanks to Paul Capestany <capestany@gmail.com> for reporting this.
 2012-02-24 13:03:24 +13:00
Aldo Cortesi
ddc9155c24 Make "~q" filter work more intuitively. 
It now matches any flow that has no response.
 2012-02-23 17:06:09 +13:00
Aldo Cortesi
2df9c52c09 Refactor filter matching. 2012-02-23 17:03:58 +13:00
Aldo Cortesi
dbd75e02f7 Create ODictCaseless for headers, use vanilla ODict for everything else. 2012-02-20 11:29:36 +13:00
Aldo Cortesi
18029df99c Use ODict for request.get_form_urlencoded and set_form_urlencoded 2012-02-20 11:13:35 +13:00
Aldo Cortesi
b0f77dfefd Unit test import cleanups. 2012-02-20 11:04:07 +13:00
Aldo Cortesi
fa11b7c9be Use ODict for Request.get_query and Request.set_query 2012-02-20 10:44:47 +13:00
Aldo Cortesi
2616f490fe Rename Headers class to ODict 
ODict is an ordered dictionary class that will be useful in many other parts of
our API.
 2012-02-20 10:39:00 +13:00
Aldo Cortesi
d32d6bc5e3 Add "p" key binding to connection list view to copy a flow. 2012-02-19 00:17:47 +13:00
Aldo Cortesi
8ddc3b4ef2 Add API for duplicating flows. 2012-02-18 23:56:40 +13:00
Aldo Cortesi
7aa79b89e8 Firm up what we consider to be a valid proxy spec. 2012-02-18 16:29:02 +13:00
Aldo Cortesi
6ad8b1a15d Firm up reverse proxy specification. 
- Extract proxy spec parsing and unparsing functions.
- Add a status indicator in mitmproxy.
- Add the "R" keybinding for changing the reverse proxy from within mitmproxy.
 2012-02-18 16:27:09 +13:00
Aldo Cortesi
a7df6e1503 Refactor reverse proxying 
- Retain the specification from the Host header as a Request's description.
- Expand upstream proxy specifications to include the scheme. We now say https://hostname:port
- Move the "R" revert keybinding to "v" to make room for a reverse proxy
binding that matches the command-line flag.
 2012-02-18 14:45:22 +13:00
Aldo Cortesi
acdc2d00b4 Repair unit tests. 2012-02-18 12:27:59 +13:00
Aldo Cortesi
da1ccfddeb 100% test coverage for flow.py 2012-02-10 15:55:58 +13:00
Aldo Cortesi
1ad7e91527 Make filter matching act more sensibly. 2012-02-10 15:31:45 +13:00
Aldo Cortesi
5f785e26b9 Add filter for detecting flows with errors. 
Also, remove dependency on weird _is_response method.
 2012-02-10 15:22:26 +13:00
Aldo Cortesi
b14c29b25c Expand test coverage. 2012-02-10 15:04:20 +13:00
Aldo Cortesi
9c985f2d20 Methods for getting and setting form urlencoded data on Request. 2012-02-10 14:27:39 +13:00
Aldo Cortesi
2709441d5b Add get_query and set_query methods to Request. 2012-02-09 16:40:31 +13:00
Aldo Cortesi
cdd5a53767 Refactor console. 
Split the console implementation out into logical components.
 2012-02-07 16:39:37 +13:00
Aldo Cortesi
35a914a549 Fix unit tests broken during previous commit. 2012-01-21 14:39:36 +13:00
Aldo Cortesi
d5e3722c97 Fix an issue caused by some editors when editing a request/response body. 
Many editors make it hard save a file without a terminating newline on the last
line. When editing message bodies, this can cause problems. For now, I just
strip the newlines off the end of the body when we return from an editor.
 2012-01-21 12:43:00 +13:00
Mark E. Haase
05111f093d Add support for filtering by HTTP method (get, post, etc.) using ~m operator. 2011-12-28 17:32:29 -05:00
Aldo Cortesi
67f2610032 Add HTTP body size limit specification to command-line tools. 2011-09-09 15:27:31 +12:00
Aldo Cortesi
28daa93268 Basic infrastructure for request and response body size limits. 2011-09-09 14:49:34 +12:00
Aldo Cortesi
e5bded7dee Improve robustness against invalid data. 2011-09-05 07:47:47 +12:00
Aldo Cortesi
45f4768a5c Add attribution and license for tnetstring.py 2011-08-19 21:53:52 +12:00
Aldo Cortesi
a566684e32 Move to typed netstrings for serialization. 
This change is backwards incompatible with the old serialization format!
 2011-08-19 21:30:24 +12:00
Aldo Cortesi
b51aac8a86 Code cleanliness - appease pychecker. 2011-08-04 10:34:34 +12:00
Aldo Cortesi
730c78ac53 Move script.Context to flow.ScriptContext 2011-08-04 10:14:44 +12:00
Aldo Cortesi
7a3b871b33 Request class now has a clean pydoc profile. 2011-08-04 09:26:26 +12:00
Aldo Cortesi
0760607a7d Further interface cleaning. 2011-08-03 23:02:33 +12:00
Aldo Cortesi
9042d3f3b9 Clean up interfaces by making some methods pseudo-private. 2011-08-03 22:48:57 +12:00
Aldo Cortesi
57c653be5f Move all HTTP objects to flow.py 
That's Request, Response, ClientConnect, ClientDisconnect, Error, and Headers.
 2011-08-03 22:41:38 +12:00
Aldo Cortesi
e337682d8e Enable "|" command to run a oneshot script on a single flow. 2011-08-03 17:35:18 +12:00
Aldo Cortesi
179cf75862 Add script hooks, enable new engine for mitmdump. 2011-08-03 16:36:20 +12:00
Aldo Cortesi
12d2b1f926 Rip out old script interface, start replacing with new stubs. 
Scripts are broken for now.
 2011-08-03 13:20:36 +12:00
Aldo Cortesi
62088a6661 Start stubbing out a much more powerful script architecture. 2011-08-03 11:06:29 +12:00
Aldo Cortesi
a817db5bd6 Refresh current connection when toggling autodecode. 
Also fix the unit tests I forgot to commit...
 2011-08-02 20:47:53 +12:00
Aldo Cortesi
1ff6a767d0 Unit test++ 2011-08-02 16:52:47 +12:00
Aldo Cortesi
357502fe03 General cleanup. 
Cut out unused variables and code, generally shut up pychecker as much as is
reasonable.
 2011-08-02 16:14:33 +12:00
Aldo Cortesi
f3742f29da We no longer need to track clientconnections. 2011-08-02 14:56:09 +12:00
Aldo Cortesi
675b3133b4 Improve performance of loading flows from a file hugely. 
Fell into the "expensive __eq__ method" trap. Oh, Python, you little scamp.
 2011-08-01 11:26:09 +12:00
Aldo Cortesi
43f1c72511 Refactor the way we calculate views of the flow list. 
The naive approach we used before recalculated the view on every access, and
consequently had serious performance problems.
 2011-08-01 11:17:01 +12:00
Aldo Cortesi
ddb5748a76 Add decoding/encoding for requests. 2011-08-01 10:43:01 +12:00
Aldo Cortesi
c89c4361c3 Merge remote-tracking branch 'alts/encoding' 2011-07-28 11:19:07 +12:00
Stephen Altamirano
78049abac1 Changes replace logic to function in both Python 2.6.x and 2.7.x 
Tests now only assume Python 2.6.x rather than requiring 2.7.x. This does not preclude the use of flags as a kwarg in replace
 2011-07-26 22:47:08 -07:00
Stephen Altamirano
acd511f676 Adds test method for Response encoding and decoding 2011-07-26 22:38:42 -07:00
Aldo Cortesi
4043829cf2 Add an eventlog option to mitmdump 
This shows client connections, disconnections and requests (before a complete
flow is assembled). We need to add an analogous display to mitmproxy.
 2011-07-23 12:57:54 +12:00
Aldo Cortesi
47e1695512 Also replace strings path for requests. 2011-07-22 20:52:13 +12:00
Aldo Cortesi
1b961fc4ad Add utility functions to search and replace strings in flows 
This is a common task in pentesting scenarios. This commit adds the following
functions:

utils.Headers.replace
proxy.Request.replace
proxy.Response.replace
flow.Flow.replace
 2011-07-22 17:48:42 +12:00
Stephen Altamirano
ecd4645988 Adds encode counterparts to decode functions 2011-07-17 20:16:47 -07:00
Aldo Cortesi
ce41046786 Refine encoding support 
- Push decoding down into the LRU cache
- Cope gracefully with corrupted data and incorrect encoding headers
 2011-07-17 10:25:25 +12:00
alts
6dc0f105cc Adds support for content encoding, namely gip and deflate 2011-07-16 02:47:06 -07:00
Aldo Cortesi
94ae720a22 Add a pretty-printing mode for urlencoded form data. 2011-07-15 16:46:54 +12:00
Aldo Cortesi
76b4c6ba82 Introduce an anti-compression command-line argument. 
This is on by default, which means we avoid compressed content unless the -z
flag is specified.
 2011-07-15 15:24:56 +12:00
Aldo Cortesi
7e21ac0eb8 Refine path completion somewhat. 
Make it match the behaviour of vim and mutt more closely
 2011-07-14 16:54:04 +12:00
Aldo Cortesi
1c9e7b982a Rewrite Headers object to preserve order and case. 2011-07-14 16:01:54 +12:00
Aldo Cortesi
8e176c2086 Cast some data read from dump files to str, to prevent unicode promotion. 
This fixes a bug that caused a traceback when de-serialized requests were
replayed. Also adds unit tests for the problem.
 2011-07-01 14:20:42 +12:00
Aldo Cortesi
18d4c3a9e9 JSON pretty-printing. 
Also rename the display modes ("pretty" instead of "indent"), and expand the
built-in documentation.
 2011-06-30 13:27:27 +12:00
Aldo Cortesi
f004326855 Try not to hang when user views large request & response bodies 
Two different strategies here:

    - Use a simple heuristic to detect if we're looking at XML data when indent
    mode is used. On non-XML data we can hang even on small documents.

    - Only view partial data for large bodies. At the moment the cutoff is
    100k. I might finetune this later.
 2011-06-27 15:59:17 +12:00
Aldo Cortesi
7d7803a4d9 Add a hideous kludge to fix not-yet-valid certificates. 
- The OpenSSL x509 has no way to explicitly set the notBefore value on
certificates.

- If two systems have the same configured time, it's possible to return a
certificate before the validity start time has arrived.

- We "solve" this by waiting for one second when a certificate is first
generated before returning the cert. The alternative is to rewrite pretty much
all of our certificate generation, a thought too horrible to contemplate.
 2011-06-11 15:16:16 +12:00
Aldo Cortesi
613e9a298e Add a new flow loading mechanism. 
We now simulate the normal connection flow when we load flows. That means
that we can run scripts, hooks, sticky cookies, etc.
 2011-05-15 11:22:35 +12:00
Aldo Cortesi
c0bd1a39e4 unit test coverage ++ 2011-03-20 18:52:16 +13:00
Aldo Cortesi
c726519e73 Add a stickyauth option. 
This allows us to replay an HTTP Authorization header, in the same way as we
replay cookies using stickycookies. This lets us conveniently get at HTTP Basic
Auth protected resources through the proxy, but is not enough to do the same
for HTTP Digest auth. We'll put that on the todo list.
 2011-03-20 17:31:54 +13:00
Aldo Cortesi
4f877cde6a Reverse order of flows in mitmproxy. 
It matches user expectations much better to have new flows appended to the bottom.
 2011-03-20 09:31:39 +13:00
Aldo Cortesi
e22fd74d06 Revamp key generation. 
We now create three different files in the .mitmproxy directory when a dummy CA
is made:

mitmproxy-ca.pem - the CA, including private key

mitmproxy-ca-cert.p12 - A pkcs12 version of the certificate, for distribution to Windows.

mitmproxy-ca-cert.pem - A PEM version of the certificate, for distribution to everyone else.
 2011-03-18 16:45:31 +13:00
Aldo Cortesi
394bd1d6b0 Interrupt interception when deleting an intercepting flow. 
Prompting the user for this is annoying.
 2011-03-15 17:53:29 +13:00
Aldo Cortesi
6d5c32ad4b Fix hang on shutdown. 2011-03-15 17:21:35 +13:00
Aldo Cortesi
fb28e71f0b Do pre-processing of requests before replay. 
This enables scripts, anticache, server playback and sticky cookies for
request replays.
 2011-03-15 15:11:03 +13:00
Aldo Cortesi
fe1e2f16ff Improve responsiveness of request and response viewing. 
- Computing the view of a large body is expensive, so we introduce an LRU cache
to hold the latest 20 results.

- Use ListView more correctly, passing it individual urwid.Text snippets,
rather than a single large one. This hugely improves render time.
 2011-03-15 13:05:33 +13:00
Aldo Cortesi
2e70a0e44b Fix unit tests on OSX when a system proxy is configured. 2011-03-14 14:05:14 +13:00
Aldo Cortesi
e35fd7126b Fix flow formatting unit tests. 2011-03-14 13:49:30 +13:00
Aldo Cortesi
1e47167ac6 Fix a traceback sometimes triggered viewing a connection with no response. 2011-03-13 20:03:13 +13:00
Aldo Cortesi
324d7c3955 Add client plaback to mitmproxy. 2011-03-13 16:50:11 +13:00
Aldo Cortesi
3ef9d2cffc flow.py 100% test coverage 2011-03-13 14:51:25 +13:00
Aldo Cortesi
c901cbbada mitmproxy prompted input now display previously set value. 
E.g. if you set a limit, then re-enter the limit prompt, you start with the
currently set value.
 2011-03-13 11:24:49 +13:00
Aldo Cortesi
9a9a2d9125 Unit tests for proxy.read_chunked 2011-03-12 16:00:01 +13:00
Aldo Cortesi
4ce393cc41 Unit tests for cmdline.py 2011-03-12 15:14:25 +13:00
Aldo Cortesi
96b10ca486 console.py 100% test coverage 2011-03-12 15:08:06 +13:00
Aldo Cortesi
fbbc43a5b2 netstring.py 100% unit test coverage. 2011-03-12 14:39:41 +13:00
Aldo Cortesi
9f16a84a9e Make mitmdump handle invalid serialized data gracefully. 2011-03-11 15:16:31 +13:00
Aldo Cortesi
7d85db0da3 Display Errors and killed connections in mitmdump. 2011-03-11 13:06:51 +13:00
Aldo Cortesi
daa9653ebe Add --norefresh to stop refreshing server playback to mitmdump. 
Also, make cookie parsing for refreshing more error-tolerant.
 2011-03-11 11:56:10 +13:00
Aldo Cortesi
e99b1d1949 Stub out refresh for server-side replay. 2011-03-09 20:05:30 +13:00
Aldo Cortesi
03f1345385 Add an --anticache option to mitmdump. 
This removes all headers that might cause a server to return 304-not-modified.

For now, all the new features are going into mitmdump - everything will be
ported over to mitmproxy once I have the feature set locked down.
 2011-03-09 13:15:31 +13:00
Aldo Cortesi
765871bd11 Store timestamps on flow components as a UTC time tuple. 
Format is:

(tm_year,tm_mon,tm_mday,tm_hour,tm_min, tm_sec,tm_wday,tm_yday,tm_isdst)
 2011-03-07 13:46:02 +13:00
Aldo Cortesi
7330f5fccf Make mitmdump server playback also exit by default. 
Like client playback, the --keepserving option makes mitmdump keep serving.
 2011-03-06 17:08:56 +13:00
Aldo Cortesi
ec00b5a66e Make mitmdump exit after client replay is complete by default. 
Add an option --keepserving to make it keep serving after replay.
 2011-03-06 16:54:49 +13:00
Aldo Cortesi
e794cbc0d8 Add flow_count and active_flow_count methods to flow.State. 
Use these in unit tests where sensible.
 2011-03-06 16:11:45 +13:00
Aldo Cortesi
42ffded626 Bump unit tests, rearrange mitmdump command-line options slightly. 2011-03-06 16:02:28 +13:00
Aldo Cortesi
bad77f3470 Add client replay options to mitmdump. 2011-03-06 11:21:31 +13:00
Aldo Cortesi
d9cb083809 Reorganize test suite to remove confusion between test utils and libmproxy utils. 2011-03-05 15:58:48 +13:00
Aldo Cortesi
5da4099ddf Massage namespace to make room for client replay. 
Mostly replay -> server_replay
 2011-03-05 13:03:26 +13:00
Aldo Cortesi
96471fde1d Basic client playback state structure. 2011-03-04 13:08:43 +13:00
Aldo Cortesi
6921b9ff2a Add an indicator that sticky cookies have been applied in mitmdump. 2011-02-25 21:23:44 +13:00