Grasscutters/mitmproxy
3
0
Fork 0
mirror of https://github.com/Grasscutters/mitmproxy.git synced 2024-11-23 16:17:49 +00:00
Code Issues Projects Releases Wiki Activity
4,857 Commits 1 Branch 67 Tags 50 MiB
6a9407d7cc
Commit Graph

639 Commits

Author SHA1 Message Date
Thomas Kriechbaumer
1c124421e3 http2: fix header_block_fragments and length 2015-06-16 15:00:28 +02:00
Thomas Kriechbaumer
d0a9d3cdda http2: only first headers frame as END_STREAM flag 2015-06-16 15:00:28 +02:00
Thomas Kriechbaumer
e3db241a2f http2: improve frame output 2015-06-16 15:00:28 +02:00
Thomas Kriechbaumer
79ff439930 add elliptic curve during TLS handshake 2015-06-16 15:00:28 +02:00
Thomas Kriechbaumer
12702b9a01 http2: improve frame output 2015-06-16 14:55:08 +02:00
Aldo Cortesi
bb206323ab Merge pull request #69 from kyle-m/master 
Adding support for upstream certificate validation when using SSL/TLS…
 2015-06-16 10:34:09 +12:00
Kyle Morton
fe764cde52 Adding support for upstream certificate validation when using SSL/TLS with an 
instance of TCPClient.
 2015-06-15 10:18:54 -07:00
Thomas Kriechbaumer
0d137eac6f simplify ALPN 2015-06-14 19:50:35 +02:00
Thomas Kriechbaumer
8d71a5b4ab http2: add authority header 2015-06-14 19:43:32 +02:00
Thomas Kriechbaumer
9c6d237d02 add new TLS methods 2015-06-14 18:17:53 +02:00
Thomas Kriechbaumer
5fab755a05 add more tests 2015-06-12 15:27:29 +02:00
Thomas Kriechbaumer
a901bc3032 http2: add response creation 2015-06-12 14:45:51 +02:00
Thomas Kriechbaumer
8ea157775d http2: general improvements 2015-06-12 14:42:07 +02:00
Thomas Kriechbaumer
eeaed93a83 improve ALPN integration 2015-06-11 15:37:17 +02:00
Thomas Kriechbaumer
0595585974 fix coding style 2015-06-08 17:00:03 +02:00
Thomas Kriechbaumer
fdbb3b76cf http2: add warning if raw data looks like HTTP/1 2015-06-08 16:54:19 +02:00
Thomas Kriechbaumer
abbe88c8ce fix non-ALPN supported OpenSSL-related tests 2015-06-08 13:25:42 +02:00
Thomas Kriechbaumer
4666d1e7bb improve ALPN support on travis 2015-06-08 12:52:06 +02:00
Thomas Kriechbaumer
359ef46905 fix coding style 2015-06-08 12:29:39 +02:00
Aldo Cortesi
9c48bfb2a5 http2: ditch the logging for now 
The API is well designed: it looks like we can get all the information
we need to expose debugging in the caller of the API.
 2015-06-06 12:30:53 +12:00
Aldo Cortesi
f2d784896d http2: resolve module structure and circular dependencies 
- Move implementation out of __init__.py to protocol.py (an anti-pattern
because it makes the kind of structural refactoring we need hard)
- protocol imports frame, frame does not import protocol. To do this, we
shift the default settings to frame. If this feels wrong, we can move
them to a separate module (defaults.py?.).
 2015-06-06 12:26:48 +12:00
Aldo Cortesi
fcaabeb455 Merge pull request #65 from Kriechi/h2-client 
HTTP/2 protocol definition
 2015-06-06 11:54:35 +12:00
Thomas Kriechbaumer
f2db8abbe8 use open instead of file 2015-06-05 20:52:11 +02:00
Thomas Kriechbaumer
fdc908cb98 http2: add protocol tests 2015-06-05 20:19:48 +02:00
Thomas Kriechbaumer
f003f87197 http2: rename module and refactor as strategy 2015-06-05 13:33:37 +02:00
Thomas Kriechbaumer
623dd850e0 http2: add logging and error handling 2015-06-05 13:33:37 +02:00
Thomas Kriechbaumer
40fa113116 http2: change header_block_fragment handling 2015-06-05 13:33:37 +02:00
Thomas Kriechbaumer
5cecbdc168 http2: add basic protocol handling 2015-06-05 13:33:37 +02:00
Thomas Kriechbaumer
e4c129026f http2: introduce state for connection objects 2015-06-05 13:33:37 +02:00
Thomas Kriechbaumer
436291764c http2: fix default settings 2015-06-05 13:33:37 +02:00
Thomas Kriechbaumer
9883509f89 simplify default ssl params for test servers 2015-06-05 13:33:37 +02:00
Aldo Cortesi
0269d0fb8b repr for websocket frames 2015-06-05 17:08:22 +12:00
Aldo Cortesi
2d9b9be1f4 Revert "tcp: clear_log to clear socket logs" 
start_log also clears the log, which is good enough.

This reverts commit 4ca62e0d9b.
 2015-06-05 11:50:29 +12:00
Aldo Cortesi
4ca62e0d9b tcp: clear_log to clear socket logs 2015-06-05 11:42:06 +12:00
Aldo Cortesi
113c5c187f Bump version 2015-06-04 11:14:47 +12:00
Maximilian Hils
c725325a78 Merge pull request #61 from Kriechi/distribute-cffi 
distribute cffi correctly
 2015-05-31 11:44:27 +02:00
Aldo Cortesi
35856ead07 websockets: nicer human readable 2015-05-31 17:24:44 +12:00
Aldo Cortesi
f7bd690e3a When we see an incomplete read with 0 bytes, it's a disconnect 
Partially fixes mitmproxy/mitmproxy:#593
 2015-05-31 17:18:55 +12:00
Aldo Cortesi
73376e605a Save first byte timestamp for writers too. 2015-05-31 16:54:14 +12:00
Aldo Cortesi
4ec181c140 Move version check to netlib, unit test it. 2015-05-31 13:12:01 +12:00
Thomas Kriechbaumer
b395049a85 distribute cffi correctly 2015-05-30 15:15:08 +02:00
Aldo Cortesi
f76bfabc5d Adjust pep8 parameters, reformat 2015-05-30 12:02:58 +12:00
Thomas Kriechbaumer
c32d8189fa cleanup imports 2015-05-29 17:00:38 +02:00
Thomas Kriechbaumer
e2de49596d add HTTP/2-capable client 2015-05-29 15:31:22 +02:00
Thomas Kriechbaumer
780836b182 add ALPN support to TCP abstraction 2015-05-29 15:31:22 +02:00
Thomas Kriechbaumer
d50b9be0d5 add generic frame parsing method 2015-05-29 15:31:22 +02:00
Thomas Kriechbaumer
4c469fdee1 add hpack to encode and decode headers 2015-05-29 15:31:22 +02:00
Thomas Kriechbaumer
754f929187 fix default argument 
Python evaluates default args during method definition.
So you get the same dict each time you call this method.
Therefore the dict is the SAME actual object each time.
 2015-05-29 11:42:46 +02:00
Thomas Kriechbaumer
5288aa3640 add human_readable() to each frame for debugging 2015-05-29 11:42:46 +02:00
Aldo Cortesi
1dda164d03 Satisfy autobots. 2015-05-28 12:18:56 +12:00
Aldo Cortesi
41af65a1c4 Merge branch 'Kriechi-cleanup' 2015-05-28 12:12:37 +12:00
Thomas Kriechbaumer
161bc2cfaa cleanup code with autoflake 
run the following command:
  $ autoflake -r -i --remove-all-unused-imports --remove-unused-variables .
 2015-05-27 11:25:33 +02:00
Thomas Kriechbaumer
e3d390e036 cleanup code with autopep8 
run the following command:
  $ autopep8 -i -r -a -a .
 2015-05-27 11:19:11 +02:00
Thomas Kriechbaumer
041ca5c499 update TLS defaults: signature hash and DH params 
* SHA1 is deprecated (use SHA256)
* increase RSA key to 2048 bits
* increase DH params to 4096 bits (LogJam attack)
 2015-05-27 10:53:57 +02:00
Thomas Kriechbaumer
d6a68e1394 remove outdated workarounds 2015-05-27 10:23:43 +02:00
Aldo Cortesi
3f25df0b12 Merge pull request #56 from Kriechi/http2-frames 
implement basic HTTP/2 frame classes
 2015-05-27 09:30:52 +12:00
Thomas Kriechbaumer
4ce6f43616 implement basic HTTP/2 frame classes 2015-05-26 17:59:29 +02:00
Aldo Cortesi
ae749975e5 Post release version bump. 2015-05-26 10:43:28 +12:00
Maximilian Hils
46fadfc823 improve displaying tcp addresses 2015-05-18 17:16:42 +02:00
Aldo Cortesi
f40bf865b1 release prep: bump version 2015-05-18 10:46:00 +12:00
Aldo Cortesi
ace4454523 Zap outdated comment 2015-05-16 11:32:18 +12:00
Aldo Cortesi
f2bc58cdd2 Add tcp.Reader.safe_read, use it in socks and websockets 
safe_read is guaranteed to raise or return a byte string of the
requested length. It's particularly useful for implementing binary
protocols.
 2015-05-05 10:47:02 +12:00
Aldo Cortesi
08b2e2a6a9 websockets: more flexible masking interface. 2015-05-01 10:31:20 +12:00
Aldo Cortesi
7d9e38ffb1 websockets: A progressive masker. 2015-05-01 10:09:35 +12:00
Aldo Cortesi
4dce7ee074 websockets: more compact and legible human_readable 2015-04-30 12:10:08 +12:00
Aldo Cortesi
8086022920 Add a tiny utility class for keeping bi-directional mappings. 
Use it in websocket and socks.
 2015-04-30 09:04:22 +12:00
Aldo Cortesi
18df329930 websockets: nicer frame construction 
- Resolve unspecified values on instantiation
- Add a check for masking key length
- Smarter resolution for masking_key and mask values. Do the right thing unless told not to.
 2015-04-24 15:42:31 +12:00
Aldo Cortesi
192fd1db7f websockets: include all header values in frame roundtrip 2015-04-24 15:31:14 +12:00
Aldo Cortesi
def93ea8ca websockets: remove validation 
We don't really need this any more. The interface is much less error prone
because bit flags are no longer integers, we have a range check on opcode on
header instantiation, and we've deferred length code calculation and so forth
into the byte render methods.
 2015-04-24 15:23:00 +12:00
Aldo Cortesi
f22bc0b4c7 websocket: interface refactoring 
- Separate out FrameHeader. We need to deal with this separately in many circumstances.
- Simpler equality scheme.
- Bits are now specified by truthiness - we don't care about the integer value.
This means lots of validation is not needed any more.
 2015-04-24 15:09:21 +12:00
Aldo Cortesi
3519871f34 websockets: refactor to avoid rundantly specifying payloads and payload lengths 2015-04-24 09:21:04 +12:00
Aldo Cortesi
bdd52fead3 websockets: extract frame header creation into a function 2015-04-24 08:47:09 +12:00
Aldo Cortesi
42a87a1d8b websockets: handshake checks only take headers 2015-04-23 08:23:51 +12:00
Aldo Cortesi
4fb49c8e55 websockets: (very) slightly nicer is_valid constraints 2015-04-21 23:49:27 +12:00
Aldo Cortesi
176e29fc09 websockets: constants, variable names, refactoring 2015-04-21 23:13:42 +12:00
Aldo Cortesi
1b509d5aea Whitespace, interface simplification 
- safe_tobytes doesn't buy us much
- move masking key generation inline
 2015-04-21 22:51:01 +12:00
Aldo Cortesi
3e0a71ea34 websockets: refactor to use http and header functions in http.py 2015-04-21 22:39:45 +12:00
Aldo Cortesi
e5f1264838 Whitespace, indentation, nounce -> nonce 2015-04-21 13:39:00 +12:00
Aldo Cortesi
7d83e388aa Whitespace, pep8, mixed indentation 2015-04-21 11:19:00 +12:00
Aldo Cortesi
dd7ea896f2 Return a named tuple from read_response 2015-04-21 11:11:16 +12:00
Aldo Cortesi
2c660d7633 Migrate requeset reading from mitmproxy to netlib 2015-04-21 11:05:12 +12:00
Chandler Abraham
4ea1ccb638 fixing test coverage, adding to_file/from_file reader writes to match socks.py 2015-04-19 22:18:30 -07:00
Aldo Cortesi
74389ef04a Websockets: reorganise 
- websockets.py to top-level
- implementations into test suite
 2015-04-20 09:38:09 +12:00
Maximilian Hils
08ba987a84 Merge branch 'master' of github.com:mitmproxy/netlib 2015-04-17 16:29:25 +02:00
Maximilian Hils
0c2ad1edb1 fix socket_close on Windows, refs mitmproxy/mitmproxy#527 2015-04-17 16:29:09 +02:00
Aldo Cortesi
7defb5be86 websockets: more whitespace, WebSocketFrame -> Frame 2015-04-17 14:29:20 +12:00
Aldo Cortesi
488c25d812 websockets: whitespace, PEP8 2015-04-17 13:57:39 +12:00
Aldo Cortesi
3bbafa24bd Merge pull request #54 from Chandler/websockets 
Netlib WebSockets take 1
 2015-04-17 13:46:51 +12:00
Aldo Cortesi
c53d89fd7f Improve flexibility of http_cookies._format_pairs 2015-04-16 08:30:54 +12:00
Aldo Cortesi
0c85c72dc4 ODict improvements 
- Setting values now tries to preserve the existing order, rather than
just appending to the end.
- __repr__ now returns  a repr of the tuple list. The old repr becomes a
.format() method. This is clearer, makes troubleshooting easier, and
doesn't assume all data in ODicts are header-like
 2015-04-15 10:28:17 +12:00
Aldo Cortesi
aeebf31927 odict: don't convert values to strings when added 2015-04-14 16:20:02 +12:00
Aldo Cortesi
d739882bf2 Add an .extend method for ODicts 2015-04-14 13:50:57 +12:00
Aldo Cortesi
6db5e0a4a1 Remove old-style set-cookie cruft, unit tests to 100% 2015-04-14 10:13:03 +12:00
Aldo Cortesi
de9e741125 Firm up cookie parsing and formatting API 
Make a tough call: we won't support old-style comma-separated set-cookie
headers. Real world testing has shown that the latest rfc (6265) is
often violated in ways that make the parsing problem indeterminate.
Since this is much more common than the old style deprecated set-cookie
variant, we focus on the most useful case.
 2015-04-14 10:02:10 +12:00
Chandler Abraham
2d72a1b6b5 100% test coverage, though still need plenty more 2015-04-13 13:36:09 -07:00
Chandler Abraham
f131f9b855 handshake tests, serialization test 2015-04-11 17:26:59 -07:00
Aldo Cortesi
1a79ef8b6c Merge branch 'master' of https://github.com/mitmproxy/netlib 2015-04-12 11:32:27 +12:00
Aldo Cortesi
2630da7263 cookies: Cater for special values, fix some bugs found in real-world testing 2015-04-12 11:30:35 +12:00
Aldo Cortesi
73ce169e3d Initial outline of a cookie parsing and serialization module. 2015-04-12 10:26:09 +12:00
Chandler Abraham
0edc04814e small cleanups, working on tests 2015-04-11 11:35:15 -07:00
Chandler Abraham
e41e5cbfdd netlib websockets 2015-04-10 18:37:41 -07:00
Maximilian Hils
e58f76aec1 fix code smell 2015-04-09 02:09:33 +02:00
Maximilian Hils
7f7ccd3a18 100% test coverage 2015-04-09 00:57:37 +02:00
Maximilian Hils
6fbe3006af fail gracefully if we cannot start a new thread 2015-04-09 00:13:01 +02:00
Maximilian Hils
d5eff70b6e fix tests on Windows 2015-03-07 01:31:31 +01:00
Maximilian Hils
dbadc1b613 clean up cert handling, fix mitmproxy/mitmproxy#472 2015-03-07 01:22:02 +01:00
Maximilian Hils
d71f3b68fd make tests more robust, fix coveralls 2015-02-27 22:27:23 +01:00
Maximilian Hils
da1eb94ccd 100% test coverage 🎉 2015-02-27 22:02:52 +01:00
Maximilian Hils
63fb433690 fix #53 2015-02-27 20:40:17 +01:00
Maximilian Hils
738a7b34a2 Merge branch 'master' of github.com:mitmproxy/netlib 2015-02-27 17:00:19 +01:00
Maximilian Hils
224f737646 add option to log ssl keys 
refs mitmproxy/mitmproxy#475
 2015-02-27 16:59:29 +01:00
Maximilian Hils
2a2402dfff ...two years is not enough. 2015-02-17 00:10:10 +01:00
Aldo Cortesi
7e5bb74e72 5 years is enough... 2015-02-17 12:03:52 +13:00
Aldo Cortesi
c9de3e770b By popular demand, bump dummy cert expiry to 5 years 
fixes #52
 2015-02-17 11:59:07 +13:00
Aldo Cortesi
3c919631d4 Bump version 2014-12-28 22:46:19 +13:00
Maximilian Hils
438c1fbc7d TCPClient: Use TLS1.1+ where available, BaseHandler: disable SSLv2 2014-12-15 12:32:36 +01:00
Aldo Cortesi
7098c90a6d Bump version to 0.11.1 2014-11-15 12:45:06 +13:00
Aldo Cortesi
c56e7a90d8 Fix tracebacks in connection finish 2014-11-15 12:31:13 +13:00
Maximilian Hils
60584387ff be more explicit about requirements 2014-11-11 12:26:20 +01:00
Aldo Cortesi
0811a9ebde .flush can raise NetlibDisconnect. This fixes a traceback found in fuzzing. 2014-11-07 16:01:41 +13:00
Aldo Cortesi
9ce2f473f6 Simplify expected_http_body_size signature, fixing a traceback found in fuzzing 2014-11-07 15:59:00 +13:00
Aldo Cortesi
ba468f12b8 Whitespace and legibility 2014-10-26 17:30:26 +13:00
Maximilian Hils
ed5e685565 refactor tcp close, fix mitmproxy/mitmproxy#376 2014-10-22 17:54:20 +02:00
Maximilian Hils
29a4e91050 fix mitmproxy/mitmproxy#375 2014-10-17 18:48:30 +02:00
Maximilian Hils
e6a8730f98 fix tcp closing for ssled connections 2014-10-09 04:42:39 +02:00
Maximilian Hils
987fa22e64 make socks reading more bulletproof 2014-10-09 01:46:08 +02:00
Maximilian Hils
9ef84ccc1c clean up code 2014-10-09 00:15:39 +02:00
Maximilian Hils
fdb6f5552d CertStore: add support for cert chains 2014-10-08 20:46:30 +02:00
Maximilian Hils
274688172d fix mitmproxy/mitmproxy#373 2014-10-08 18:40:46 +02:00
Maximilian Hils
e73a2dbab1 minor changes 2014-09-28 03:15:26 +02:00
Aldo Cortesi
0e30796469 Short-form getstate 2014-09-17 14:04:26 +12:00
Aldo Cortesi
414a0a1602 Adjust for state object protocol changes in mitmproxy. 2014-09-17 11:47:07 +12:00
Aldo Cortesi
b21df0cf44 Merge branch 'master' of ssh.github.com:cortesi/netlib 2014-09-09 10:10:10 +12:00
Aldo Cortesi
63c1efd394 Remove avoidable imports from OpenSSL 
Fixes #38
 2014-09-09 10:08:56 +12:00
Maximilian Hils
f90ea89e69 more verbose errors 2014-09-08 18:38:05 +02:00
Aldo Cortesi
5dcc7f78df Merge pull request #34 from bbaetz/master 
Change the criticality of a number of X509 extentions, to match
 2014-09-07 12:50:36 +12:00
Aldo Cortesi
754b627937 Merge pull request #48 from mitmproxy/mitmproxy_issue_341 
Adjust netlib to mitmproxy changes
 2014-09-07 12:47:49 +12:00
Aldo Cortesi
f4013dcd40 Add a FIXME note for discarded credentials 2014-09-07 12:47:17 +12:00
Aldo Cortesi
52f430c934 Merge pull request #37 from pritambaral/urlparse 
More accurate host, port parsing
 2014-09-07 12:46:01 +12:00
Aldo Cortesi
f98989b075 Merge branch 'master' into mitmproxy_issue_341 
Conflicts:
	netlib/tcp.py
 2014-09-07 12:39:59 +12:00
Aldo Cortesi
b688661ffb Merge branch 'func' 2014-09-07 11:25:28 +12:00
Aldo Cortesi
3b81d678c4 Use print function after future import 2014-09-07 11:24:41 +12:00
Maximilian Hils
c2e74ef95c Merge branch 'master' into mitmproxy_issue_341 2014-09-04 21:21:03 +02:00
Maximilian Hils
d9a731b23a make inequality comparison work 2014-09-04 19:18:43 +02:00
Maximilian Hils
4bf7f3c0ff set source_address if not manually specified 2014-09-04 16:55:02 +02:00
Maximilian Hils
ec628bc37d fix tcp.Address inequality comparison 2014-09-04 01:10:44 +02:00
Maximilian Hils
3d489f3bb7 adapt netlib.wsgi to changes in mitmproxy/mitmproxy#341 2014-09-03 17:15:50 +02:00
Maximilian Hils
6d1b601ddf minor cleanups 2014-08-16 15:53:07 +02:00
Maximilian Hils
1c1167eda0 use passlib instead of md5crypt 2014-08-16 15:28:09 +02:00
Maximilian Hils
1b8529649c Merge pull request #42 from mitmproxy/stream 
Stream
 2014-07-31 22:05:44 +02:00
kronick
197dae9183 Made attribute optional (as it is in pyOpenSSL) 
See 0d7e8a1af2 -- It looks like this constant isn't set on some platforms (including Raspberry Pi's libssl)
 2014-07-29 15:12:13 +02:00
Maximilian Hils
254a686235 Merge branch 'master' into stream 
Conflicts:
	netlib/http.py
 2014-07-21 14:02:56 +02:00
Maximilian Hils
6bd5df79f8 refactor response length handling 2014-07-21 14:01:24 +02:00
Maximilian Hils
d382bb27bf certstore: add support for asterisk form to DNTree replacement 2014-07-19 00:02:31 +02:00
Maximilian Hils
a7837846a2 temporarily replace DNTree with a simpler cert lookup mechanism, fix mitmproxy/mitmproxy#295 2014-07-18 22:55:25 +02:00
Brad Peabody
280d9b8625 added some additional functions for dealing with chunks - needed for mitmproxy streaming capability 2014-07-17 22:34:29 -07:00
Maximilian Hils
24ef9c61a3 improve docs 2014-07-14 17:38:49 +02:00
Brad Peabody
273c25a705 added option for read_response to only read the headers, beginnings of implementing streamed result in mitmproxy 2014-07-12 22:42:06 -07:00
Maximilian Hils
4d5d8b6511 mark nsCertType non-critical, fix #39 2014-06-29 13:10:07 +02:00
Maximilian Hils
e69133f98c remove ntop windows workaround 2014-06-25 21:16:47 +02:00
Maximilian Hils
6405595ae8 socks module: polish, add tests 2014-06-25 20:31:28 +02:00
Maximilian Hils
dc3d3e5f0a add inet_ntop/inet_pton functions 2014-06-25 20:31:10 +02:00
Maximilian Hils
217660f5db add socks module 2014-06-25 14:30:42 +02:00
Pritam Baral
dc071c4ea7 Ignore username:password part in url 2014-05-28 07:10:10 +05:30
Maximilian Hils
66ac56509f add support for ctx.load_verify_locations, refs mitmproxy/mitmproxy#174 2014-05-21 01:14:55 +02:00
Maximilian Hils
52c6ba8880 properly subclass Exception in HTTPError 2014-05-15 18:15:29 +02:00
Maximilian Hils
71834aeab1 make cert and key mandatory 2014-05-15 14:15:33 +02:00
Maximilian Hils
a8345af282 extract cert creation to be accessible in handle_sni callbacks 2014-05-15 13:51:59 +02:00
Maximilian Hils
92081eee04 Update certutils.py 
refs mitmproxy/mitmproxy#200
 2014-04-25 19:40:37 +02:00
Maximilian Hils
c2c952b3cc make error message example less abstract. 2014-03-31 12:44:20 +02:00
Pedro Worcel
bb10dfc505 Instead of removing the error, for consistency, leaving the error as-was 
and replaced the message with something that may or may not be more
understandable :P
 2014-03-31 20:19:23 +13:00
Pedro Worcel
e7c3e4c5ac Change error into awesome user-friendlyness 
Hi there,

I was getting a very weird error "ODict valuelist should be lists", when attempting to add a header.

My code was as followed:

```
        msg.headers["API-Key"] = new_headers["API-Key"]                                                                                                                                                                              
 42         msg.headers["API-Sign"] = new_headers["API-Sign"]
```

In the end, that was because there could be multiple equal headers. In order to cater to that, it you guys might enjoy the patch I attach, for it converts strings automatically into lists of multiple headers.

I think it should work, but I haven't tested it :$

It'd allow me to have the above code, instead of this one below:

```
        msg.headers["API-Key"] = [new_headers["API-Key"]]                                                                                                                                                                               
 42         msg.headers["API-Sign"] = [new_headers["API-Sign"]]
```
 2014-03-30 20:58:47 +13:00
Bradley Baetz
d8f54c7c03 Change the criticality of a number of X509 extentions, to match 
the RFCs and real-world CAs/certs.

This improve compatability with older browsers/clients.
 2014-03-20 11:12:11 +11:00
Maximilian Hils
34e469eb55 create dhparam file if it doesn't exist, fix mitmproxy/mitmproxy#235 2014-03-11 20:23:27 +01:00
Maximilian Hils
4bd15a28b7 fix #28 2014-03-10 17:43:39 +01:00
Aldo Cortesi
f5cc63d653 Certificate flags 2014-03-10 17:29:27 +13:00
Aldo Cortesi
2a12aa3c47 Support Ephemeral Diffie-Hellman 2014-03-07 16:38:50 +13:00
Aldo Cortesi
52b14aa1d1 CertStore: cope with certs that have no common name 2014-03-05 17:29:14 +13:00
Aldo Cortesi
86730a9a4c Handler convert_to_ssl now takes a key object, not a path. 2014-03-05 13:43:52 +13:00
Aldo Cortesi
0c3bc1cff2 Much more sophisticated certificate store 
- Handle wildcard lookup
- Handle lookup of SANs
- Provide hooks for registering override certs and keys for specific
domains (including wildcard specifications)
 2014-03-05 13:19:16 +13:00
Aldo Cortesi
7c82418e0b Beef up CertStore, add DH params. 2014-03-04 14:12:58 +13:00
Aldo Cortesi
cfaa3da25c Use PyOpenSSL's underlying ffi interface to get current cipher for connections. 2014-03-02 21:37:28 +13:00
Aldo Cortesi
1acaf1c880 Re-add state operations to ODict. 2014-03-02 16:54:21 +13:00
Aldo Cortesi
e381c03668 Cleanups, tests, and no-cover directives for code sections we can't test. 2014-03-02 16:47:10 +13:00
Aldo Cortesi
7788391903 Minor improvement to CertStore interface 2014-03-02 13:50:19 +13:00
Aldo Cortesi
3443bae94e Cipher suite selection for client connections, improved error handling 2014-02-27 18:35:16 +13:00
Maximilian Hils
c276b4294c allow super() on TCPServer, add thread names for better debugging 2014-02-15 23:16:28 +01:00
Maximilian Hils
a72ae4d85c Bump version 
Do it now already so that mitmproxy will warn the user if netlib is not from master.
 2014-02-11 12:09:58 +01:00
Aldo Cortesi
3d52d16e8d Merge branch 'tcp_proxy' 2014-02-07 10:50:23 +13:00
Maximilian Hils
7fc544bc7f adjust netlib.wsgi to reflect changes in mitmproxys flow format 2014-02-05 21:34:14 +01:00
Maximilian Hils
0bbc40dc33 store used sni in TCPClient, add equality check for tcp.Address 2014-02-04 04:51:41 +01:00
Maximilian Hils
dc45b4bf19 move StateObject back into libmproxy 2014-01-31 01:06:53 +01:00
Maximilian Hils
ff9656be80 remove subclassing of tuple in tcp.Address, move StateObject into netlib 2014-01-30 20:07:30 +01:00
Maximilian Hils
e18ac4b672 re-add server attribute to BaseHandler 2014-01-28 20:30:16 +01:00
Maximilian Hils
763cb90b66 add tcp.Address to unify ipv4/ipv6 address handling 2014-01-28 17:26:35 +01:00
Aldo Cortesi
8266699acd Silence pyflakes, adjust requirements.txt 2014-01-19 18:17:06 +13:00
Maximilian Hils
71c1017575 Merge branch 'master' into tcp_proxy 2014-01-18 22:55:51 +01:00
Maximilian Hils
0f22039bca add CONNECT request to list of request types that don't have a response body 2014-01-18 22:55:40 +01:00
Maximilian Hils
d0a6d2e254 fix tests, remove duplicate code 2014-01-09 05:33:21 +01:00
Maximilian Hils
b0b93d1c3e Merge remote-tracking branch 'origin/master' into tcp_proxy 2014-01-09 01:57:50 +01:00
Maximilian Hils
951f2d517f change parameter names to reflect changes 2014-01-09 01:57:37 +01:00
Aldo Cortesi
ac1a700fa1 Make certificate not-before time 48 hours. 
Fixes #200
 2014-01-08 14:46:55 +13:00
Aldo Cortesi
1c6f714193 Merge pull request #26 from mitmproxy/refactor_read_http_body 
refactor http.read_http_body
 2014-01-04 14:37:34 -08:00
Aldo Cortesi
5717e7300c Make it possible to pass custom environment variables into wsgi apps. 2014-01-05 10:57:50 +13:00
Maximilian Hils
cebec67e08 refactor read_http_body 2013-12-15 06:43:54 +01:00
Maximilian Hils
f2e8efdf15 merge smurfix/ipv6, add ipv6 support for TCPServer, add ipv6 test 2013-12-13 15:04:38 +01:00
Maximilian Hils
969595cca7 add requirements.txt, small changes 2013-12-13 06:24:08 +01:00
Matthias Urlichs
6f26cec83e tab fix 2013-12-12 07:11:13 +01:00
Matthias Urlichs
a7ac97eb82 support ipv6 2013-12-12 07:00:58 +01:00
Aldo Cortesi
d66fd5ba1b Bump version 2013-12-10 22:20:12 +13:00
Aldo Cortesi
4840c6b3bf Fix race condition in test suite. 2013-12-08 15:26:30 +13:00
Maximilian Hils
64139a1e7e merge origin/master 2013-12-08 01:39:50 +01:00
Maximilian Hils
390f2a46c9 make AuthAction generic 2013-12-08 01:37:45 +01:00
Aldo Cortesi
7213f86d49 Unit test auth actions. 2013-12-08 13:35:42 +13:00
Aldo Cortesi
d05c20d8fa Domain checks for persistent cert store is now irrelevant. 
We no longer store these on disk, so we don't care about path
components.
 2013-12-08 13:15:08 +13:00
Aldo Cortesi
98a580cf69 Merge pull request #19 from rouli/ciphersuites 
adding cipher list selection option to BaseHandler
 2013-12-07 15:51:44 -08:00
Aldo Cortesi
af8f98d493 Merge pull request #22 from fictivekin/custom-o-cn 
allow specification of o, cn, expiry
 2013-12-07 15:42:54 -08:00
Aldo Cortesi
ed74b62856 Merge branch 'fix_invalid_tcp_close' 2013-12-08 10:15:43 +13:00
Aldo Cortesi
5aad09ab81 Fix client certificate request feature. 2013-12-08 10:15:19 +13:00
Aldo Cortesi
bed2aed9db Merge branch 'master' of ssh.github.com:cortesi/netlib 2013-11-21 13:09:11 +13:00
Maximilian Hils
e402e3b862 add custom argparse actions to seamlessly integrate ProxyAuth classes 2013-11-21 01:07:56 +01:00
Maximilian Hils
643602c066 Merge branch 'fix_windows_bugs' into fix_invalid_tcp_close 2013-11-19 05:03:10 +01:00
Maximilian Hils
5e4ccbd7ed attempt to fix #24 2013-11-19 04:11:24 +01:00
Aldo Cortesi
07e970346f Merge branch 'master' of ssh.github.com:cortesi/netlib 2013-10-18 08:17:39 +13:00
Sean Coates
642b3f002e remove tempfile and shutil imports because they're not actually used 2013-10-07 16:55:35 -04:00
Sean Coates
53b7c5abdd allow specification of o, cn, expiry 2013-10-07 16:48:30 -04:00
Paul
98f765f693 Don't create a certificate request when creating a dummy cert 2013-09-24 21:18:41 +02:00
Aldo Cortesi
8a261b2c01 Bump version. 2013-08-25 10:30:48 +12:00
Aldo Cortesi
7428f95474 Handle interrupted system call errors. 2013-08-25 10:22:09 +12:00
Israel Nir
d5b3e397e1 adding cipher list selection option to BaseHandler 2013-08-21 13:42:30 +03:00
Maximilian Hils
28a0030c1e compatibility fixes for windows 2013-08-19 19:41:20 +02:00
Maximilian Hils
c44f354fd0 fix windows bugs 2013-08-17 16:15:37 +02:00
Aldo Cortesi
62edceee09 Revamp dummy cert generation. 
We no longer use on-disk storage - we just keep the certs in memory.
 2013-08-12 16:03:29 +12:00
Aldo Cortesi
2da57ecff0 Correct order of precedence for SSL errors. 2013-08-11 11:47:07 +12:00
Aldo Cortesi
b9f06b473c Better handling of cert errors. 2013-08-10 23:07:09 +12:00
Aldo Cortesi
f5fdfd8a9f Clarify the interface for flush and close methods. 2013-07-30 09:42:13 +12:00
Aldo Cortesi
6709253629 Merge pull request #16 from mitmproxy/fix_socket_buffer 
attempt to fix 'half-duplex' TCP close sequence
 2013-07-28 14:55:40 -07:00
Andrey Plotnikov
02376b6a75 Add socket binding support for TCPClient 2013-07-07 13:33:56 +08:00
Maximilian Hils
68e2e782b0 attempt to fix 'half-duplex' TCP close sequence 2013-06-17 17:03:17 +02:00
Aldo Cortesi
73f8a1e2e0 Bump version. 2013-06-16 13:38:39 +12:00
Maximilian Hils
c9ab1c60b5 always read files in binary mode 2013-06-16 00:28:21 +02:00
Aldo Cortesi
7f0aa415e1 Add a request_client_cert argument to server SSL conversion. 
By default, we now do not request the client cert. We're supposed to be able to
do this with no negative effects - if the client has no cert to present, we're
notified and proceed as usual.  Unfortunately, Android seems to have a bug
(tested on 4.2.2) - when an Android client is asked to present a certificate it
does not have, it hangs up, which is frankly bogus.  Some time down the track
we may be able to make the proper behaviour the default again, but until then
we're conservative.
 2013-05-13 08:48:21 +12:00
Aldo Cortesi
9c13224353 Fix exception hierarchy. 2013-05-05 13:49:20 +12:00
Tim Becker
241465c368 extensions aren't supported in v1, set to v3 (value=2) if using them. 2013-04-19 15:37:14 +02:00
Aldo Cortesi
a94d17970e Sync version number with mitmproxy. 2013-03-05 09:09:52 +13:00
Aldo Cortesi
5f0ad7b2a6 Ensure that HTTP methods are ASCII. 2013-03-03 22:13:23 +13:00
Aldo Cortesi
5a050bb6b2 Tighten up checks on port ranges and path character sets. 2013-03-03 21:39:15 +13:00
Aldo Cortesi
b21a7da142 parse_url: Handle invalid IPv6 addresses 2013-03-03 15:12:58 +13:00
Aldo Cortesi
7b9300743e More parse_url solidification: check that port is in range 0-65535 2013-03-03 15:08:17 +13:00
Aldo Cortesi
cd4ed8530f Check that hosts in parse_url do not contain NULL bytes. 2013-03-03 15:03:57 +13:00
Aldo Cortesi
2897ddfbee Stricter error checking for http.parse_url 2013-03-03 14:52:06 +13:00
Aldo Cortesi
1fe1a802ad 100% test coverage. 2013-03-03 12:16:09 +13:00
Aldo Cortesi
0acab862a6 Integrate HTTP auth, test to 100% 2013-03-03 10:37:28 +13:00
Aldo Cortesi
97537417f0 Factor out http.parse_response_line 2013-03-02 16:57:38 +13:00
Aldo Cortesi
0fa6351965 ODict.keys 2013-02-28 09:28:48 +13:00
Aldo Cortesi
f30df13384 Make sni_handler an argument to BaseHandler.convert_to_ssl 2013-02-25 21:11:09 +13:00
Aldo Cortesi
97e11a219f Housekeeping and cleanup, some minor argument name changes. 2013-02-24 15:36:15 +13:00
Aldo Cortesi
c6f9a2d74d More accurate description of an HTTP read error, make pyflakes happy. 2013-02-24 11:08:43 +13:00
Aldo Cortesi
7d18535665 100% test coverage 2013-01-27 19:21:18 +13:00
Aldo Cortesi
7433dfceae Bump unit tests, fix two serious wee buglets discovered. 2013-01-26 21:29:45 +13:00
Aldo Cortesi
e5b125eec8 Introduce the mock module to improve unit tests. 
There are a few socket corner-cases that are incredibly hard to reproduce in a
unit test suite, so we use mock to trigger the exceptions instead.
 2013-01-26 21:19:35 +13:00
Aldo Cortesi
cc4867064b Streamline netlib.test API 2013-01-25 16:03:59 +13:00
Aldo Cortesi
2eb6651e51 Extract TCP test utilities into netlib.test 2013-01-25 15:54:41 +13:00
Aldo Cortesi
7248a22d5e Improve error signalling for client certificates. 2013-01-20 22:36:54 +13:00
Aldo Cortesi
00d20abdd4 Beef up client certificate handling substantially. 2013-01-20 22:13:38 +13:00
Aldo Cortesi
1499529e62 Fix client cert typo. 2013-01-18 17:07:35 +13:00
Rouli
04048b4c73 renaming the timestamp in preparation of other timestamps that will be added later, adding tests 2013-01-16 22:30:19 +02:00
Israel Nir
10457e876a adding read timestamp to enable better resolution of when certain reads were performed (timestamp is updated when the first byte is available on the network) 2013-01-10 15:51:37 +02:00
Aldo Cortesi
e4acace8ea Sanity-check certstore common names. 2013-01-06 01:34:39 +13:00
Aldo Cortesi
91834ea78f Generate certificates with a commencement date an hour in the past. 
This helps smooth over small discrepancies in client and server times, where
it's possible for a certificate to seem to be "in the future" to the client.
 2013-01-06 01:16:58 +13:00
Aldo Cortesi
72032d7fe7 Basic certificate store implementation and cert utils API cleanup. 2013-01-06 01:16:25 +13:00
Aldo Cortesi
d3b46feb60 Handle non-integer port error in parse_init_connect correctly 2013-01-05 20:06:55 +13:00
Aldo Cortesi
ddc08efde1 Minor cleanup of http.parse_init* methods. 2013-01-04 14:23:52 +13:00
Maximilian Hils
043d05bcde add __iter__ for odict 2012-12-05 04:03:39 +01:00
Aldo Cortesi
f8e10bd6ae Bump version. 2012-10-31 22:26:09 +13:00
Aldo Cortesi
6517d9e717 More info on disconnect exception. 2012-10-14 09:03:23 +13:00
Aldo Cortesi
77869634e2 Limit reads to block length. 2012-10-09 16:25:15 +13:00
Aldo Cortesi
15679e010d Add a settimeout method to tcp.BaseHandler. 2012-10-01 11:30:02 +13:00
Aldo Cortesi
064b4c8001 Make cleanBin escape carriage returns. 
We get confusing output on terminals if we leave \r unescaped.
 2012-09-27 10:59:46 +12:00
Aldo Cortesi
b308824193 Create netlib.utils, move cleanBin and hexdump from libmproxy.utils. 2012-09-24 11:21:48 +12:00
Aldo Cortesi
3a21e28bf1 Split FileLike into Writer and Reader, and add logging functionality. 2012-09-24 11:10:21 +12:00
Aldo Cortesi
8a6cca530c Don't create fresh FileLike objects when converting to SSL 2012-09-24 10:47:41 +12:00
Aldo Cortesi
1c80c2fdd7 Add a collection of standard User-Agent strings. 
These will be used in both mitmproxy and pathod.
 2012-09-01 23:04:44 +12:00
Aldo Cortesi
33557245bf v0.2.1 2012-08-23 12:57:22 +12:00
Aldo Cortesi
877a3e2062 Add a get_first convenience function to ODict. 2012-08-18 18:14:13 +12:00
Aldo Cortesi
1c21a28e64 read_headers: handle some crashes, return None on invalid data. 2012-07-30 12:50:35 +12:00
Aldo Cortesi
eafa5566c2 Handle disconnects on flush. 2012-07-30 11:30:31 +12:00
Aldo Cortesi
4fb5d15f14 Bump version. 2012-07-29 15:53:42 +12:00
Aldo Cortesi
728ef107a0 Ignore SAN entries that we don't understand. 2012-07-24 14:55:54 +12:00
Aldo Cortesi
91752990d5 Handle HTTP responses that have a body but no content-length or transfer encoding 
We check if the server sent a connection:close header, and read till the socket
closes.

Closes #2
 2012-07-24 11:41:18 +12:00
Aldo Cortesi
eb88cea3c7 Catch an amazingly subtle SSL connection corruption bug. 
Closing a set of pseudo-file descriptors in the wrong order caused junk data to
be written to the SSL stream. An apparent bug in OpenSSL then lets this corrupt
the _next_ SSL connection.
 2012-07-23 23:20:32 +12:00
Aldo Cortesi
ed64b0e796 Fix http_protocol parsing crash discovered with pathoc fuzzing. 2012-07-22 12:35:16 +12:00
Aldo Cortesi
619f3c6edc Handle unexpected SSL connection termination in readline. 2012-07-21 20:51:05 +12:00
Aldo Cortesi
b2c491fe39 Handle socket disconnects on reads. 2012-07-21 17:50:21 +12:00
Aldo Cortesi
29f907ecf9 Handle HTTP versions malformed due to non-integer major/minor numbers. 2012-07-21 17:27:23 +12:00
Aldo Cortesi
2387d2e8ed Timeout for TCP clients. 2012-07-21 16:10:54 +12:00
Aldo Cortesi
ba53d2e4ca Set ssl_established right after the connection object is changed. 2012-07-20 15:15:07 +12:00
Aldo Cortesi
a1a1663c0f Fix cert path. 2012-07-20 14:45:58 +12:00
Aldo Cortesi
0791fe6cc6 Merge branch 'master' of ssh.github.com:cortesi/netlib 2012-07-20 14:44:23 +12:00
Aldo Cortesi
63d789109a close() methods for clients and servers. 2012-07-20 14:43:51 +12:00
Maximilian Hils
9ab7842c81 fix relative certdir 2012-07-11 11:09:41 +02:00
Aldo Cortesi
1227369db3 Signal errors back to caller in WSGI .serve() 2012-07-11 07:16:45 +12:00
Aldo Cortesi
4fdc2179e2 Don't write empty values. 2012-07-10 16:34:39 +12:00
Aldo Cortesi
721e2c8277 Somewhat nicer handling of errors after thread termination. 2012-07-10 16:22:45 +12:00
Aldo Cortesi
ba7437abcb Add an exception to indicate remote disconnects. 2012-07-08 23:50:38 +12:00
Aldo Cortesi
20cc1b6aa4 Refactor TCP test suite. 2012-07-05 10:57:20 +12:00
Aldo Cortesi
96af5c16a0 Expose SSL options, use TLSv1 by default for client connections. 2012-07-04 21:30:07 +12:00
Aldo Cortesi
67669a2a57 Allow control of buffer size for TCPClient, improve error messages. 2012-06-30 10:52:28 +12:00
Aldo Cortesi
7480f87cd7 Add utility function for converstion to PEM. 2012-06-28 14:56:21 +12:00
Aldo Cortesi
3f9aad53ab Return a certutils.SSLCert object from get_remote_cert. 2012-06-28 10:59:03 +12:00
Aldo Cortesi
bae86480d4 Merge branch 'master' of github.com:cortesi/netlib 2012-06-28 09:57:33 +12:00
Aldo Cortesi
92c7d38bd3 Handle obscure termination scenario, where interpreter exits before thread termination. 2012-06-28 09:56:58 +12:00
Aldo Cortesi
a1491a6ae0 Add a get_remote_cert method to tcp client. 2012-06-28 08:15:55 +12:00
Aldo Cortesi
b0ef9ad07b Refactor certutils.SSLCert API. 2012-06-27 22:11:58 +12:00
Aldo Cortesi
f7fcb1c80b Add certutils to netlib. 2012-06-27 16:42:00 +12:00
Aldo Cortesi
97071c0952 Merge branch 'master' of ssh.github.com:cortesi/netlib 2012-06-27 16:24:34 +12:00
Aldo Cortesi
5d4c7829bf Minor refactoring. 2012-06-27 16:24:22 +12:00
Aldo Cortesi
d0fd8385e6 Fix termiantion error in file read. 2012-06-27 12:11:55 +12:00
Aldo Cortesi
abe335e57d Add a flag to track SSL connection establishment. 2012-06-26 23:52:35 +12:00
Aldo Cortesi
658c9c0446 Hunt down a tricky WSGI socket hang. 2012-06-26 14:49:23 +12:00
Aldo Cortesi
ccf2603ddc Add SNI. 2012-06-26 09:50:42 +12:00
Aldo Cortesi
ea457fac2e Perform handshake immediately on SSL conversion. 
Otherwise the handshake happens at first write, which can balls up if either
side hangs immediately.
 2012-06-25 16:16:01 +12:00
Aldo Cortesi
353efec7ce Improve TCPClient interface. 
- Don't pass SSL parameters on instantiation.
- Add a convert_to_ssl method analogous to that in TCPServer.
 2012-06-25 14:42:15 +12:00
Aldo Cortesi
47f862ae27 Add a finished flag to BaseHandler, and catch an extra OpenSSL exception. 2012-06-25 11:34:10 +12:00
Aldo Cortesi
f3237503a7 Don't connect during __init__ methods for either client or server. 
This means we now need to do these things explicitly at the caller.
 2012-06-25 11:23:04 +12:00
Aldo Cortesi
8f0754b9c4 SSL tests, plus some self-signed test certificates. 2012-06-25 11:00:39 +12:00
Aldo Cortesi
7d01d5c797 Don't read all from server by default. 
This can cause us to hang waiting for data. More research is needed to
establish the right course of action here.
 2012-06-24 23:13:09 +12:00
Aldo Cortesi
820ac5152e WSGI SERVER_PORT should be a string. 2012-06-24 22:57:09 +12:00
Aldo Cortesi
5988b65419 Add and unit test http.read_response 2012-06-24 22:45:40 +12:00
Aldo Cortesi
0de765f360 Make read_headers return an ODictCaseless object. 2012-06-24 21:49:23 +12:00
Aldo Cortesi
171de05d8e Add http_status.py 2012-06-23 18:34:51 +12:00
Aldo Cortesi
1263221ddd 100% testcoverage for netlib.http 2012-06-23 15:07:42 +12:00
Aldo Cortesi
5cf6aeb926 protocol.py -> http.py 2012-06-23 13:56:17 +12:00
Aldo Cortesi
227e72abf4 README, setup.py, version 2012-06-23 13:49:57 +12:00
Aldo Cortesi
b706200796 Drop default poll interval to 0.1s. 2012-06-20 11:01:40 +12:00
Aldo Cortesi
084be7684d Close socket on shutdown. 2012-06-20 10:51:02 +12:00
Aldo Cortesi
ce1ef55456 Adapt WSGI, convert test suite to nose. 2012-06-19 14:23:22 +12:00
Aldo Cortesi
c7e9051cbb Import wsgi. 2012-06-19 10:42:25 +12:00
Aldo Cortesi
b558997fd9 Initial checkin. 2012-06-19 09:42:32 +12:00