Commit Graph

268 Commits

Author SHA1 Message Date
Thomas Kriechbaumer
b732a1cb3d reimplement streaming for HTTP/1 2015-08-01 14:37:33 +02:00
Thomas Kriechbaumer
54866c0faa revert hard-coded HTTP/2 protocol changes 2015-08-01 14:37:33 +02:00
Thomas Kriechbaumer
db02553e2a move code from mitmproxy to netlib 2015-08-01 14:37:32 +02:00
Thomas Kriechbaumer
4f38c6b90e attach application protocol to connection 2015-07-30 19:53:29 +02:00
Thomas Kriechbaumer
a5d9e1f44d move code to netlib and implement protocols 2015-07-30 13:53:17 +02:00
Thomas Kriechbaumer
89f22f7359 refactor connection & protocol handling 2015-07-30 13:53:17 +02:00
Maximilian Hils
d2ae6b630e Merge pull request #548 from macmantrl/no_serverconn
Prevent unnecessary upstream server connects
2015-07-26 11:58:21 +02:00
Thomas Kriechbaumer
e9bbcfadc1 fix module imports 2015-07-25 20:00:34 +02:00
Thomas Kriechbaumer
cedc0407de refactor to use netlib.http protocols 2015-07-22 15:30:12 +02:00
Thomas Kriechbaumer
bb265d0c40 use new netlib module names 2015-07-22 15:30:11 +02:00
Thomas Kriechbaumer
df1b0df39f use netlib.http_semantics for generic data 2015-07-22 15:30:11 +02:00
Terry Long
4c50c36345 Prevent unecessary upstream server connects
Selectively connect to upstream server based on no_upstream_cert
option. When no_upstream_cert is used during server replay, prevent
connecting to the upstream server unless absolutely necessary.
2015-07-03 07:55:37 -07:00
Aldo Cortesi
876252eba8 Remove stray print & massage whitespace a bit 2015-06-25 10:27:22 +12:00
iroiro123
fbb23b5c9f changed error handling (ssl spoof mode) 2015-06-23 01:49:22 +09:00
iroiro123
fd90367329 SSL Spoof mode 2015-06-21 00:51:56 +09:00
iroiro123
378aa78324 Spoof mode 2015-06-20 21:43:50 +09:00
iroiro123
59ec291b6c HTTP Transparent Proxy 2015-06-18 23:53:27 +09:00
Aldo Cortesi
7890450b0c Handle invalid IDNA encoding in hostnames
Fixes #622
2015-06-12 16:00:16 +12:00
Aldo Cortesi
a05a70d816 Add coding style check, reformat. 2015-05-30 12:03:28 +12:00
Aldo Cortesi
14341ffa19 Merge pull request #584 from dlethin/feature/http_1_0
Make sure proxy returns the httpversion specified in the request
2015-05-18 11:03:05 +12:00
Doug Lethin
cdff79fd4c Make sure proxy returns the httpversion specified in the request rather
than hardcoding to 1.1.
2015-05-15 22:20:09 -04:00
Nick Raptis
202ede3c54 Accurately timestamp start of request
When building a request from a stream, try to get an accurate
start timestamp from the Reader. This was already in the code
and also used when building response objects, but was ommited
in commit ddf458b330

Without his logic and when the client is reusing a connection
to send requests, the timestamp_start of subsequent requests
is early and equal to when the connection started read blocking
2015-05-08 10:58:05 +03:00
Aldo Cortesi
1c26516b18 pretty_size now lives in netlib.utils 2015-04-30 12:18:01 +12:00
Aldo Cortesi
37d731aacd Missed some un-needed variable declarations 2015-04-21 11:42:27 +12:00
Aldo Cortesi
ddf458b330 HTTP request reading moves to netlib 2015-04-21 11:05:37 +12:00
Aldo Cortesi
5ff4303124 Adjust for ODict interface change 2015-04-15 10:29:57 +12:00
Aldo Cortesi
c335c2b533 Add set_cookies method to HTTPResponse 2015-04-14 16:23:51 +12:00
Aldo Cortesi
ab7e2857cc New get_cookies for HttpResponse 2015-04-14 15:14:36 +12:00
Aldo Cortesi
e17eacd8d7 New get_cookie and set_cookie implementations for HTTPRequest 2015-04-14 13:45:38 +12:00
Aldo Cortesi
937a358aa9 Merge branch 'master' of ssh.github.com:mitmproxy/mitmproxy 2015-04-14 12:01:37 +12:00
Aldo Cortesi
bea0bd236a Housekeeping and cleanups
- No output to stdout on load in examples - they muck up the test suite.
- Use the odict module directly, rather than aliasing it. The small convenience
this gives to scripters is not worth it.
- Move the cookie tests from the flow test module to the protocol_http test
module.
2015-04-14 11:58:10 +12:00
Maximilian Hils
ab2b98b988 fix pretty_host if no host is present 2015-04-10 19:35:42 +02:00
Maximilian Hils
6852eb9d0a fix #553 2015-04-10 14:59:38 +02:00
Maximilian Hils
423530fc25 Merge pull request #515 from krzysiekbielicki/master
[#514] Add support for ignoring payload params in multipart/form-data
2015-03-26 18:16:31 +01:00
Maximilian Hils
02a61ea45d structure components 2015-03-21 22:49:51 +01:00
Krzysztof Bielicki
244ef243d7
[#514] Add support for ignoring payload params in multipart/form-data 2015-03-10 10:44:06 +01:00
Maximilian Hils
e1b6cf9401 fix #319 2015-02-27 15:24:27 +01:00
Youhei Sakurai
5916767e03 Correct typo; https://github.com/mitmproxy/mitmproxy/issues/319 2015-02-27 10:22:27 +09:00
Youhei Sakurai
10f81e596b Change from checking __call__ to using callable;
https://github.com/mitmproxy/mitmproxy/issues/319
2015-02-27 10:15:07 +09:00
Youhei Sakurai
77bb0b74ed Maybe it should work; https://github.com/mitmproxy/mitmproxy/issues/319 2015-02-27 02:44:47 +09:00
Marcelo Glezer
010b921a93 added sni and ssl_established=true in ServerConnection. removed check for None value of server_conn in http.py 2015-02-12 13:41:58 -03:00
Marcelo Glezer
1df78f75c5 set sni to None when no server_conn is None 2015-02-10 17:18:23 -03:00
Maximilian Hils
53fadd05f4 fix #451 2015-02-07 16:26:19 +01:00
Maximilian Hils
52d696df3d fix ssl shutdown in tcp mode, refs #464 2015-02-05 16:46:55 +01:00
Maximilian Hils
310fb18aac fix #464 2015-02-05 15:24:32 +01:00
Maximilian Hils
1f454b577f web: CSP, revert functionality, serve content 2014-12-25 16:10:47 +01:00
Maximilian Hils
7ed6f10e35 web: much ui work, such christmas 2014-12-25 02:03:55 +01:00
Maximilian Hils
1c4f4c2494 web: duplicate, delete, replay flows 2014-12-24 01:07:57 +01:00
Maximilian Hils
2861d99de4 web: intercept feature 2014-12-23 20:33:42 +01:00
Maximilian Hils
4e635d7a6f allow specification of SSL version, only allow TLS1.0+ by default 2014-12-15 12:46:13 +01:00
Aldo Cortesi
01fa5d3f07 Merge pull request #414 from mitmproxy/flowviews2
Flowviews2
2014-12-12 22:08:15 +13:00
Maximilian Hils
b95f0c9971 fix #411 2014-12-08 17:17:37 +01:00
Maximilian Hils
5b1fefee9b add inline script example for websocket passthrough, fix #340 2014-12-01 03:04:48 +01:00
Maximilian Hils
992536c2bc make header processing configurable by inline scripts, refs #340 2014-12-01 02:28:03 +01:00
Maximilian Hils
3887e7ed29 fix error html 2014-11-26 04:56:17 +01:00
Maximilian Hils
ed8249023f introduce revised views, port over changes from multiple_views branch 2014-11-26 04:18:21 +01:00
Maximilian Hils
ec23594191 add sni support to LiveConnection.change_server 2014-11-19 01:27:20 +01:00
Maximilian Hils
0c52b4e3b9 handle script hooks in replay, fix tests, fix #402 2014-11-14 00:26:22 +01:00
Maximilian Hils
c3ec551546 fix #401 2014-11-07 09:52:46 +01:00
Wade 524
ce18cd8ba4 Fixing issue #392. 2014-10-31 11:50:03 -07:00
Aldo Cortesi
340d0570bf Legibility 2014-10-26 18:32:45 +13:00
Aldo Cortesi
16654ad6a4 Fix crash while streaming
Found using fuzzing. Reproduction with pathoc, given "mitmproxy -s" and
pathod running on 9999:

get:'http://localhost:9999/p/':s'200:b\'foo\':h\'Content-Length\'=\'3\'':i58,'\x1a':r

 return flow.FlowMaster.run(self)
   File "/Users/aldo/mitmproxy/mitmproxy/libmproxy/controller.py", line 111, in run
     self.tick(self.masterq, 0.01)
   File "/Users/aldo/mitmproxy/mitmproxy/libmproxy/flow.py", line 613, in tick
     return controller.Master.tick(self, q, timeout)
   File "/Users/aldo/mitmproxy/mitmproxy/libmproxy/controller.py", line 101, in tick
     self.handle(*msg)
   File "/Users/aldo/mitmproxy/mitmproxy/libmproxy/controller.py", line 118, in handle
     m(obj)
   File "/Users/aldo/mitmproxy/mitmproxy/libmproxy/flow.py", line 738, in handle_responseheaders
     self.stream_large_bodies.run(f, False)
   File "/Users/aldo/mitmproxy/mitmproxy/libmproxy/flow.py", line 155, in run
     r.headers, is_request, flow.request.method, code
   File "/Users/aldo/mitmproxy/mitmproxy/netlib/http.py", line 401, in expected_http_body_size
     raise HttpError(400 if is_request else 502, "Invalid content-length header: %s" % headers["content-length"])
 netlib.http.HttpError: Invalid content-length header: ['\x1a3']
2014-10-26 17:58:36 +13:00
Aldo Cortesi
7aee9a7c31 Spacing and legibility 2014-10-26 17:44:49 +13:00
Maximilian Hils
6cef6fbfec tweak SSL detection heuristics 2014-10-21 15:08:39 +02:00
Maximilian Hils
e114858438 add generic tcp proxying, fix #374 2014-10-18 18:29:35 +02:00
Maximilian Hils
f04693c047 fix typo 2014-10-08 21:41:03 +02:00
Maximilian Hils
6ce6b1ad69 replay: carry over SNI value 2014-10-02 00:58:40 +02:00
Maximilian Hils
4167713cc0 fix timestamp formatting: end_time may not be set 2014-09-22 17:01:06 +02:00
Maximilian Hils
60cec1f9b6 clean up timestamp handling 2014-09-21 22:46:48 +02:00
Maximilian Hils
b0374710e4 start to fill detailpane 2014-09-19 01:35:36 +02:00
Maximilian Hils
0d64cc9327 flowtable: add selection indicator, add keyboard navigation 2014-09-18 02:22:10 +02:00
Maximilian Hils
102bd07568 implement FlowStore basics 2014-09-17 17:30:19 +02:00
Aldo Cortesi
b4ecd96beb Introduce short form object state, and connect the ends to send data to web app 2014-09-17 13:58:56 +12:00
Aldo Cortesi
d998790c2f Clean up and clarify StateObject
- Flatten the class hierarchy
- get_state, load_state, from_state are public
- Simplify code
- Remove __eq__ and __neq__. This fundamentally changes the semantics of
inherited objects in a way that's not part of the core function of the
class
2014-09-17 11:41:42 +12:00
Aldo Cortesi
b9531ac89b Whitespace & formatting 2014-09-17 09:54:17 +12:00
Aldo Cortesi
f7da58ca9b Basic websocket connection, code cleanup. 2014-09-17 09:40:25 +12:00
Maximilian Hils
fbc4bb38df mitmproxy -> libmproxy.main. do some leanup on the way 2014-09-08 23:34:43 +02:00
Maximilian Hils
ebd539b49f update examples, fix #353 2014-09-08 17:03:34 +02:00
Maximilian Hils
d06b4bfa4e remove proxy mode abstraction: always be clear which mode we are in 2014-09-08 14:32:42 +02:00
Maximilian Hils
5a2a5760d0 improve logging 2014-09-08 13:34:08 +02:00
Maximilian Hils
ac27d1236f improve change_server api, add example how to change the upstream server 2014-09-07 18:01:37 +02:00
Maximilian Hils
5226ac200f update code docs, make sure that url attribute never returns prettified/formatted values 2014-09-07 03:18:18 +02:00
Maximilian Hils
32e1ed212d streamline HTTPHandler.handle_flow() 2014-09-05 19:39:05 +02:00
Maximilian Hils
2a6337343a update docs, mostly revert 2f44b26b4c 2014-09-05 15:16:20 +02:00
Maximilian Hils
a7a3b5703a change replay_request behaviour, refs #346; test upstream proxy mode 2014-09-05 00:18:17 +02:00
Maximilian Hils
795e19f6b7 coverage++ 2014-09-04 16:37:50 +02:00
Maximilian Hils
f4d4332472 coverage++ 2014-09-04 14:46:25 +02:00
Maximilian Hils
1d45c54a04 Merge branch 'issue_341' into server_change_api 2014-09-04 14:18:21 +02:00
Maximilian Hils
1135666ee6 Merge branch 'master' into issue_341 2014-09-04 14:18:07 +02:00
Maximilian Hils
39fa579dd5 remove outdated IOError test case (f5fdfd8a9f) 2014-09-04 14:15:49 +02:00
Maximilian Hils
4781c565a9 fix #344 2014-09-04 12:29:44 +02:00
Maximilian Hils
2f44b26b4c improve HTTPRequest syntax 2014-09-03 23:44:54 +02:00
Maximilian Hils
cd43c5ba9c simplify server changes for inline scripts 2014-09-03 20:12:30 +02:00
Maximilian Hils
0c3b6ee667 fix IOError handling 2014-09-03 17:33:08 +02:00
Maximilian Hils
b0cfeff06d fix #341 - work on flows instead of request/response internally. 2014-09-03 16:57:56 +02:00
Maximilian Hils
951a6fcc36 http: ignore disconnects that happen between requests 2014-09-03 12:31:39 +02:00
Maximilian Hils
712e5860aa ui: display * instead of 0.0.0.0 for the bound address 2014-09-03 12:24:14 +02:00
Maximilian Hils
1e4e332ef9 improve error handling 2014-09-02 18:13:18 +02:00
Maximilian Hils
1a41c15c03 minor fixes 2014-08-31 16:52:26 +02:00
Maximilian Hils
1f47f7b6b2 refactor protocol handling, fix #332 2014-08-30 20:15:19 +02:00
Maximilian Hils
b97b1f17cf fix #328 2014-08-24 14:22:11 +02:00
Maximilian Hils
e4d6089f9a fix #330 2014-08-23 13:53:43 +02:00
Maximilian Hils
3d861d7a67 make sure that flow.server_conn is set when script hook is called 2014-08-18 01:54:45 +02:00
Maximilian Hils
5b7e19a77e fix #327 2014-08-18 00:55:30 +02:00
Maximilian Hils
623cbd4e51 fix: changes to request.host in the request inline script hook are now considered for the server connection 2014-08-17 23:59:44 +02:00
Maximilian Hils
ecf8081ba6 implement passthrough mode, fixes #175 2014-08-09 03:03:21 +02:00
Maximilian Hils
f4a1459ebe fix #274 2014-08-09 01:02:58 +02:00
Maximilian Hils
c88613f596 fix replay form 2014-08-09 00:53:16 +02:00
Maximilian Hils
2e6a698287 fix #308 2014-08-08 19:04:58 +02:00
Maximilian Hils
4d2109ef92 fix display of urls in upstream mode, refs #308 2014-08-08 17:49:02 +02:00
Maximilian Hils
a9e6121a08 properly express state information on server connections, refs #315 2014-08-08 02:45:24 +02:00
Maximilian Hils
13f030ccb5 suppress SPDY/HTTP2 announcement headers, fix #277 2014-08-03 16:25:38 +02:00
Maximilian Hils
3133136da7 fix #307 2014-08-03 02:34:29 +02:00
Maximilian Hils
e9401a2123 Merge branch 'stream'
Conflicts:
	libmproxy/protocol/http.py
2014-07-31 22:29:13 +02:00
Maximilian Hils
c9be57b682 fix #313 2014-07-27 03:28:23 +02:00
Maximilian Hils
2b31a9c49e refactor tcp handling, fix #280 2014-07-27 02:39:17 +02:00
Maximilian Hils
83536e7e53 minor fixes 2014-07-27 02:10:39 +02:00
Maximilian Hils
e9c834a30d fix #259 2014-07-26 12:02:18 +02:00
Maximilian Hils
4382829b7d workaround: always make sure that flow.response.reply exists 2014-07-25 18:47:48 +02:00
Maximilian Hils
d9ac029ec7 always initialize HTTPResponse.stream attribute 2014-07-23 00:21:33 +02:00
Maximilian Hils
4b4a18a2e4 add --stream options, various fixes 2014-07-21 21:06:55 +02:00
Maximilian Hils
562ac9e721 unify stream handling 2014-07-21 14:09:24 +02:00
Maximilian Hils
7398db80db simplify responseheader scripthook 2014-07-20 11:17:53 +02:00
Maximilian Hils
0ba5a2cf19 Merge branch 'stream' of https://github.com/bradleypeabody/mitmproxy into stream 2014-07-20 10:57:51 +02:00
Brad Peabody
560e23af09 fixed handling of Transfer-Encoding header during streaming; wrote tests for streaming support 2014-07-19 19:10:14 -07:00
Brad Peabody
c47ddaa3a0 basic attempt to implement streaming response, needs testing 2014-07-17 22:43:26 -07:00
Maximilian Hils
71ba7089e2 fix parameter naming 2014-07-14 17:26:22 +02:00
Brad Peabody
ca7d398b42 make include_content=False work by passing this on to netlib's http.read_response 2014-07-12 22:40:51 -07:00
Maximilian Hils
84a016dccf expose sockname 2014-06-25 23:03:15 +02:00
Maximilian Hils
00fd243810 fix up error messages 2014-05-15 18:16:42 +02:00
Maximilian Hils
aab071380a beef up error messages 2014-03-26 00:51:18 +01:00
Maximilian Hils
3a89c84eaf add requests to requrements, improve error logging 2014-03-19 17:18:21 +01:00
Maximilian Hils
28f82d052d fix display bugs (fixes #243) 2014-03-18 19:38:02 +01:00
Maximilian Hils
4d9799f103 add support for log levels 2014-03-13 01:04:45 +01:00
Maximilian Hils
15c82f743f fix temporary server change, coverage++ 2014-03-11 02:16:22 +01:00
Maximilian Hils
c1fff51b1b always use relative imports, fix status bar bug 2014-03-10 22:36:47 +01:00
Maximilian Hils
e6349b540f split up protocol\__init__.py 2014-03-10 21:57:50 +01:00
Maximilian Hils
9cc10630c8 update docs, fix #215 2014-03-10 17:01:30 +01:00
Maximilian Hils
fe58c1c6eb add advanced proxying options, add SSL-terminating capability to mitmproxy 2014-03-10 05:11:51 +01:00
Maximilian Hils
78750a8b4d lay the foundations for --(in|out)(abs|rel) command line switches, as proposed in https://groups.google.com/forum/#!topic/mitmproxy/nApno2TXS0c 2014-03-10 02:32:27 +01:00
Maximilian Hils
5598a8de82 finish proxy.py split up 2014-03-09 21:51:24 +01:00
Maximilian Hils
fc4fe83eaf split up proxy.py 2014-03-09 21:13:08 +01:00
Maximilian Hils
2b01c4eee7 Merge branch 'master' of github.com:mitmproxy/mitmproxy 2014-03-08 15:47:27 +01:00
Maximilian Hils
3032672f10 combine asterisk-form and origin-form into relative form 2014-03-08 15:47:09 +01:00
Aldo Cortesi
32af668814 Minor cleanups. 2014-03-02 22:15:53 +13:00
Maximilian Hils
c2828de4a1 fix bug: ensure that server connection is present for tcp connections 2014-02-17 17:25:45 +01:00
Maximilian Hils
d9a6037299 fix comments 2014-02-09 02:54:07 +01:00
Aldo Cortesi
a85974eaa8 Fix flow initialization order error
Resolves #210
2014-02-08 17:17:35 +13:00
Aldo Cortesi
b642b4870b Better response when mitmproxy crashes. 2014-02-08 14:15:19 +13:00
Maximilian Hils
735e4400c4 add tests for reconnect to upstream proxy, ensure that server_reconnect is always hooked 2014-02-07 18:14:15 +01:00