Grasscutters/mitmproxy
3
0
Fork 0
mirror of https://github.com/Grasscutters/mitmproxy.git synced 2024-11-23 16:17:49 +00:00
Code Issues Projects Releases Wiki Activity
1,077 Commits 1 Branch 67 Tags 50 MiB
e5d2829364
Commit Graph

218 Commits

Author SHA1 Message Date
Aldo Cortesi
32ad26f8bf Add a size() method to flow.Request and flow.Response. 2012-08-04 13:18:05 +12:00
Aldo Cortesi
31a092f6b4 Minor refactoring. Make stop_stream also close the associated file descriptor. 2012-07-27 00:19:18 +12:00
Aldo Cortesi
b4e9e55c34 Be more tolerant of corrupted or truncated flows. 
We load as far as possible. mitmproxy will only terminate if it was not able to
recover any flows. mitmdump will stop loading as soon as an error is
encountered, but not exit with an error.
 2012-07-24 15:15:41 +12:00
Aldo Cortesi
01b8b0d876 Refine semantics of replay_request method. 2012-07-10 23:29:33 +12:00
Aldo Cortesi
aab45078ad Unit test request replay thread. 
This is a small patch, but is the culmination of lots of work: we can now unit
test the deep innards of mitmproxy, with coverage. There's a lot more to come
in this vein.
 2012-07-09 11:03:55 +12:00
Aldo Cortesi
572e8a4962 Add streaming to FlowMaster 2012-07-09 10:18:37 +12:00
Aldo Cortesi
aa708a2d28 Fix error when serializing reverted SSL flows. 2012-07-05 11:52:56 +12:00
Aldo Cortesi
4acc9aca27 Firm up handling of Unicode data 
- Modify GridEditor to know about the destination encoding of data
- Ensure that get_url always returns ASCII
 2012-07-05 11:27:40 +12:00
Aldo Cortesi
9c30e2e86d Correct handing of IDNA encoding of internationalized domain names. 
- Use IDNA encoding for hostnames gleaned by upstream-cert sniffing
- Use IDNA decoding for URL display in mitmproxy and mitmdump.
 2012-07-03 22:27:16 +12:00
Aldo Cortesi
35fdd16940 Serialize address as part of ClientConnect objects. 
This is a serialization format change!
 2012-07-01 00:16:30 +12:00
Aldo Cortesi
d74a341e5d Beef up logging substantially. 2012-07-01 00:15:03 +12:00
Aldo Cortesi
47651b1ff2 Serialization and de-serialization of new cert format. 2012-06-28 14:29:15 +12:00
Aldo Cortesi
243e0efefc Adjust for new get_remote_cert API. 2012-06-28 10:02:14 +12:00
Aldo Cortesi
35ee0c098f Remove certutils from mitmproxy. 2012-06-27 16:43:33 +12:00
Aldo Cortesi
874649f134 Adapt for API changes in netlib. 2012-06-23 14:06:34 +12:00
Aldo Cortesi
1b1ccab8b7 Extract protocol and tcp server implementations into netlib. 2012-06-19 09:58:50 +12:00
Aldo Cortesi
7b9756f48e Refactor protocol.py to remove dependence on flow and utils. 2012-06-17 10:52:39 +12:00
Aldo Cortesi
c7952371b7 Fix a problem in ODictCaseless that could cause duplicate headers. 2012-06-15 17:40:08 +12:00
Aldo Cortesi
d032504b17 Fix an exception when replaying a flow with no response. 2012-06-10 17:10:43 +12:00
Aldo Cortesi
236447c65f Pass server HTTP version back to clients. 2012-06-10 13:29:09 +12:00
Aldo Cortesi
6ba5f0f35b Add HTTP version to response objects. 
Another change in the serialization format.
 2012-06-10 13:27:43 +12:00
Aldo Cortesi
52779d9db9 Refactoring of proxy.py 
- Correctly pass HTTP request version on to upstream servers
- Adjust tests not to hang due to a pathod response with no content-length
 2012-06-10 13:17:18 +12:00
Aldo Cortesi
55ddf853cd Add HTTP version to flow.Request 
This is a serialization format change, that makes us incompatible with previous
versions.
 2012-06-10 10:46:22 +12:00
Aldo Cortesi
b7b357528c Port mitmproxy test suite entirely to nose. 2012-06-09 13:42:43 +12:00
Aldo Cortesi
ee2950cd19 Fix a crashing bug when replacing text in a flow with unicode bodies. 2012-05-25 18:10:31 -07:00
Aldo Cortesi
0a90a3eaba Refuse to replay a request with missing content. 2012-05-16 18:24:32 +12:00
Aldo Cortesi
0c2d894cea Add the ability to flag content as missing in a request or a response. 
We'll use this in a number of situations. First, we'll soon have response
streaming that directly pipes responses to clients. These will be content-less
from mitmproxy's perspective. Second, we'll be growing new events that fire
after headers are received, but before content is read.
 2012-05-16 15:42:58 +12:00
Aldo Cortesi
2fe54d17df Don't specify Content-Length on empty content. 
Sometimes, mitmproxy would specify a content-length header value of 0 when
content was empty. Some rare servers (like piratebay.org) would barf on this.
 2012-04-25 14:38:20 +12:00
Aldo Cortesi
c8d2b2594b Add a WSGI adapter that lets us serve a WSGI app out of mitmproxy. 
This commit adds:
    - A WSGI App adapter for mitmproxy
    - An app registry in the proxy instance that lets us link WSGI apps with
    (hostname, port) combinations.
    - Fixes for a number of bugs discovered while creating this feature.
 2012-04-24 14:52:29 +12:00
Aldo Cortesi
8c96264304 Serialized data version check. 2012-04-11 10:10:53 +12:00
Aldo Cortesi
79a0334a02 Improve revert model 
- Flows are backed up whenever an interactive, non-script change is made.
- That backup is canonical and never changed - "V" will always revert to it.

This makes more sense than what we had previously....
 2012-04-04 09:47:57 +12:00
Aldo Cortesi
ab0e10e60f Serialize requestcount for ClientConnect objects. 2012-04-03 22:37:24 +12:00
Aldo Cortesi
4979a22d3e Add accessor method for SSLCert object on Response. 2012-04-02 17:02:23 +12:00
Aldo Cortesi
ab1d8fa350 Expand SSL cert support 
- Capture the remote SSL certificate
- Expose the remote cert as an attribute on Response
- Expand the certutils.SSLCert interface to expose more cert info
 2012-04-02 16:21:23 +12:00
Aldo Cortesi
62e51018d0 Refactor pretty view mechanism. 
Also start adding unit tests for this subsystem.
 2012-03-24 14:20:24 +13:00
Aldo Cortesi
ed74ed24a0 Add error indications to GridEditor. 2012-03-23 13:28:33 +13:00
Aldo Cortesi
5690e7c399 Generalize GridEditor to N columns. 
Start adding a replacement rule editor.
 2012-03-18 14:39:21 +13:00
Aldo Cortesi
c8ae1e85b3 Hooks -> ReplaceHooks 
It makes more sense to specialize this, which will let me build a nicer
interface for replacement hooks in mitmproxy.
 2012-03-17 11:31:05 +13:00
Aldo Cortesi
08f410cacc Add a hooks mechanism, based on filter expressions. 2012-03-16 17:13:11 +13:00
Aldo Cortesi
d138af7217 replace() methods now decode and re-encode contents before substitution. 2012-03-16 11:24:18 +13:00
Aldo Cortesi
d51b8cab0c Add a decoded context manager. 
This simplifies a common chore when modifying traffic - decoding the object,
modifying it, then re-encoding it with the same encoding afterwards. You can
now simply say:

with flow.decoded(request):
    request.content = "bar"
 2012-03-16 11:12:56 +13:00
Aldo Cortesi
8d662e6636 Set a "unique" serial number for each generated cert. 2012-03-14 11:20:25 +13:00
Valtteri Virtanen
ed56d67cea Adds no-pop option to server-side replay 2012-03-05 11:05:11 +02:00
Aldo Cortesi
986a41d180 Unit test++. 2012-02-25 12:19:54 +13:00
Aldo Cortesi
25fa596cd6 Fix detection of URL-encoded forms. 
Thanks to Paul Capestany <capestany@gmail.com> for reporting this.
 2012-02-24 13:03:24 +13:00
Aldo Cortesi
2df9c52c09 Refactor filter matching. 2012-02-23 17:03:58 +13:00
Aldo Cortesi
554047da85 License notifications, minor docs. 2012-02-23 15:52:01 +13:00
Aldo Cortesi
4f38b3a9c0 Documentation and screenshots. 2012-02-22 17:17:13 +13:00
Aldo Cortesi
a4270efaf2 Always return an ODict from get_query 2012-02-21 13:00:45 +13:00
Aldo Cortesi
dbd75e02f7 Create ODictCaseless for headers, use vanilla ODict for everything else. 2012-02-20 11:29:36 +13:00
Aldo Cortesi
18029df99c Use ODict for request.get_form_urlencoded and set_form_urlencoded 2012-02-20 11:13:35 +13:00
Aldo Cortesi
fa11b7c9be Use ODict for Request.get_query and Request.set_query 2012-02-20 10:44:47 +13:00
Aldo Cortesi
2616f490fe Rename Headers class to ODict 
ODict is an ordered dictionary class that will be useful in many other parts of
our API.
 2012-02-20 10:39:00 +13:00
Aldo Cortesi
25a06c3ec1 Minor doc fixes and import cleanups. 2012-02-20 10:15:58 +13:00
Aldo Cortesi
77a33c441b Add duplicate_flow and replay_request hooks to ScriptContext. 2012-02-19 11:29:49 +13:00
Aldo Cortesi
d32d6bc5e3 Add "p" key binding to connection list view to copy a flow. 2012-02-19 00:17:47 +13:00
Aldo Cortesi
8ddc3b4ef2 Add API for duplicating flows. 2012-02-18 23:56:40 +13:00
Aldo Cortesi
6ad8b1a15d Firm up reverse proxy specification. 
- Extract proxy spec parsing and unparsing functions.
- Add a status indicator in mitmproxy.
- Add the "R" keybinding for changing the reverse proxy from within mitmproxy.
 2012-02-18 16:27:09 +13:00
Aldo Cortesi
a7df6e1503 Refactor reverse proxying 
- Retain the specification from the Host header as a Request's description.
- Expand upstream proxy specifications to include the scheme. We now say https://hostname:port
- Move the "R" revert keybinding to "v" to make room for a reverse proxy
binding that matches the command-line flag.
 2012-02-18 14:45:22 +13:00
Aldo Cortesi
14def89f50 Fix a problem in deserialization of flows with errors. 2012-02-18 12:25:22 +13:00
Aldo Cortesi
1ad7e91527 Make filter matching act more sensibly. 2012-02-10 15:31:45 +13:00
Aldo Cortesi
5f785e26b9 Add filter for detecting flows with errors. 
Also, remove dependency on weird _is_response method.
 2012-02-10 15:22:26 +13:00
Aldo Cortesi
b14c29b25c Expand test coverage. 2012-02-10 15:04:20 +13:00
Aldo Cortesi
5326b7610a Enable editing of urlencoded form data with KVEditor. 2012-02-10 14:35:23 +13:00
Aldo Cortesi
9c985f2d20 Methods for getting and setting form urlencoded data on Request. 2012-02-10 14:27:39 +13:00
Aldo Cortesi
2709441d5b Add get_query and set_query methods to Request. 2012-02-09 16:40:31 +13:00
Aldo Cortesi
4b9ee4c31e Very basic KV editor mockup. 2012-02-06 09:49:49 +13:00
Aldo Cortesi
3b246f7e27 Simple fix for a unicode error when editing a request URL. 2011-10-26 14:49:15 +13:00
Aldo Cortesi
d9db1cf5b3 Change size limit cmdline flag to -Z, enable size limits for replay. 2011-09-09 17:31:36 +12:00
Aldo Cortesi
67f2610032 Add HTTP body size limit specification to command-line tools. 2011-09-09 15:27:31 +12:00
Aldo Cortesi
4ac59a7859 Fix a rare crash in sticky cookies. 2011-08-26 18:03:03 +12:00
Aldo Cortesi
8fbba59e8d Fix a problem with sticky cookie domain matching. 
Just like everything else cookie-related in the standard library,
cookielib.domain_match is fucked up.
 2011-08-26 17:37:12 +12:00
Aldo Cortesi
45f4768a5c Add attribution and license for tnetstring.py 2011-08-19 21:53:52 +12:00
Aldo Cortesi
a566684e32 Move to typed netstrings for serialization. 
This change is backwards incompatible with the old serialization format!
 2011-08-19 21:30:24 +12:00
András Veres-Szentkirályi
b1dc418a53 Replaced unnecessary lists with generators 2011-08-18 23:29:57 +02:00
Aldo Cortesi
f23818ceea Add a "done" event for scripts. 
Called exactly once after all other events.
 2011-08-05 14:08:03 +12:00
Aldo Cortesi
87623a8d75 Rip out autodecode 
We simplify things as follows:

    - If we're in "pretty" view mode, we autodecode.
    - Otherwise, we display raw data, and the user can manually encode/decode
    with z shortcut.
 2011-08-04 10:54:42 +12:00
Aldo Cortesi
b51aac8a86 Code cleanliness - appease pychecker. 2011-08-04 10:34:34 +12:00
Aldo Cortesi
730c78ac53 Move script.Context to flow.ScriptContext 2011-08-04 10:14:44 +12:00
Aldo Cortesi
1662b8505b Clean pydoc profile for flow.Flow 2011-08-04 09:56:44 +12:00
Aldo Cortesi
8ef208a9e2 Clean pydoc profile for flow.Response, flow.Error 2011-08-04 09:44:48 +12:00
Aldo Cortesi
7a3b871b33 Request class now has a clean pydoc profile. 2011-08-04 09:26:26 +12:00
Aldo Cortesi
0760607a7d Further interface cleaning. 2011-08-03 23:02:33 +12:00
Aldo Cortesi
9042d3f3b9 Clean up interfaces by making some methods pseudo-private. 2011-08-03 22:48:57 +12:00
Aldo Cortesi
57c653be5f Move all HTTP objects to flow.py 
That's Request, Response, ClientConnect, ClientDisconnect, Error, and Headers.
 2011-08-03 22:41:38 +12:00
Aldo Cortesi
e337682d8e Enable "|" command to run a oneshot script on a single flow. 2011-08-03 17:35:18 +12:00
Aldo Cortesi
179cf75862 Add script hooks, enable new engine for mitmdump. 2011-08-03 16:36:20 +12:00
Aldo Cortesi
f7e4e89b12 Move the event notification mechanism into flow.py 2011-08-03 13:33:18 +12:00
Aldo Cortesi
12d2b1f926 Rip out old script interface, start replacing with new stubs. 
Scripts are broken for now.
 2011-08-03 13:20:36 +12:00
Aldo Cortesi
8cc0469ee7 Tweak encoding behaviour 
- Don't fail to identity encoding when an unknown encoding is specified.
- Don't constrain encodings. I want to try to modify traffic as little as
possible by default.
- When decoding, delete content-encoding header rather than set it to "identity"
- Refuse to decode/encode when there is an existing but unknown
content-encoding header.
 2011-08-02 20:42:46 +12:00
Aldo Cortesi
357502fe03 General cleanup. 
Cut out unused variables and code, generally shut up pychecker as much as is
reasonable.
 2011-08-02 16:14:33 +12:00
Aldo Cortesi
f3742f29da We no longer need to track clientconnections. 2011-08-02 14:56:09 +12:00
Aldo Cortesi
675b3133b4 Improve performance of loading flows from a file hugely. 
Fell into the "expensive __eq__ method" trap. Oh, Python, you little scamp.
 2011-08-01 11:26:09 +12:00
Aldo Cortesi
43f1c72511 Refactor the way we calculate views of the flow list. 
The naive approach we used before recalculated the view on every access, and
consequently had serious performance problems.
 2011-08-01 11:17:01 +12:00
Stephen Altamirano
78049abac1 Changes replace logic to function in both Python 2.6.x and 2.7.x 
Tests now only assume Python 2.6.x rather than requiring 2.7.x. This does not preclude the use of flags as a kwarg in replace
 2011-07-26 22:47:08 -07:00
Aldo Cortesi
e6288e2d07 Fix crash when sticky cookies are read from file. 
Cookielib expects strings, not unicode.
 2011-07-24 16:08:27 +12:00
Aldo Cortesi
1b961fc4ad Add utility functions to search and replace strings in flows 
This is a common task in pentesting scenarios. This commit adds the following
functions:

utils.Headers.replace
proxy.Request.replace
proxy.Response.replace
flow.Flow.replace
 2011-07-22 17:48:42 +12:00
Stephen Altamirano
74d8b18408 Removes should_autodecode attribute from Response. Adds commandline option 'd' to toggle autodecode, adds togglable option 'd' to do the same 2011-07-21 20:22:13 -07:00
alts
6dc0f105cc Adds support for content encoding, namely gip and deflate 2011-07-16 02:47:06 -07:00
Aldo Cortesi
76b4c6ba82 Introduce an anti-compression command-line argument. 
This is on by default, which means we avoid compressed content unless the -z
flag is specified.
 2011-07-15 15:24:56 +12:00