Grasscutters/mitmproxy
3
0
Fork 0
mirror of https://github.com/Grasscutters/mitmproxy.git synced 2024-11-30 11:19:23 +00:00
Code Issues Projects Releases Wiki Activity
5,409 Commits 1 Branch 67 Tags 50 MiB
85bf5e18f6
Commit Graph

706 Commits

Author SHA1 Message Date
Maximilian Hils
fa45722ed8 Merge pull request #1402 from mhils/console-fixes 
minor fixes
 2016-07-23 12:49:57 -07:00
Maximilian Hils
fcb906dc97 improve dumper addon text alignment (#1415) 2016-07-23 12:01:05 -07:00
Maximilian Hils
61de6fa1d6 fix test_view_urlencoded 2016-07-23 11:55:27 -07:00
Maximilian Hils
798759d2b3 fix content view cache invalidation 2016-07-21 19:49:32 -07:00
Aldo Cortesi
02acfb1242 Fix netlib.utils.is_valid_host 
- Don't crash when passed an empty string. This translated into an actual core
crash, discovered while fuzzing with afl.
- Taking a slice of length one out of bytes returns an integer, so the check
for trailing period in this function never worked on Python3.
- Add unit tests.
 2016-07-21 10:38:37 +12:00
Maximilian Hils
b27db1fc81 minor fixes 2016-07-19 18:09:33 -07:00
Thomas Kriechbaumer
a32312cecc fix-1366 2016-07-16 18:31:37 +02:00
Maximilian Hils
b27d59095d Merge pull request #1306 from mitmproxy/message-body-encoding 
Improve Message Body Encoding
 2016-07-16 00:13:58 -07:00
Maximilian Hils
e6e39ce80f preserve content-type parameter order 2016-07-15 23:46:12 -07:00
Aldo Cortesi
1a1f7e6fd9 replacehooks -> addon 
Also fixes a bug in header replacements in netlib that resulted in a mutable
multidict.
 2016-07-16 18:37:30 +12:00
Maximilian Hils
3602fd7a36 Merge remote-tracking branch 'origin/master' into message-body-encoding 2016-07-15 23:17:57 -07:00
Maximilian Hils
a3c7c84d49 improve message content semantics 2016-07-15 22:50:33 -07:00
Aldo Cortesi
b94f5fd361 Convert examples and example tests for new-style scripts 
Remove the test that just loads all the example scripts for now - it's a very
low-value test, and we need to think of something better.
 2016-07-14 19:54:15 +12:00
Thomas Kriechbaumer
c90de8b9a4 fix sysinfo for py3 2016-07-10 13:16:23 +02:00
Shadab Zafar
39f5108400 Test cookies.is_expired separately 2016-07-10 01:06:50 +05:30
Shadab Zafar
c92992f03b Move cookie expiry detection to separate function 2016-07-09 22:20:25 +05:30
Shadab Zafar
83a1cc5a9a Make escape_control_characters handle strings on Py2 2016-07-09 13:28:20 +05:30
Shadab Zafar
0a21d27004 strutils.hexdump returns native_strings 2016-07-09 12:43:21 +05:30
Maximilian Hils
0a1431ed2c fix #1314 2016-07-08 15:50:11 -07:00
Maximilian Hils
7a5b21556b fix tests 2016-07-07 19:34:57 -07:00
Maximilian Hils
00dce24015 tests++ 2016-07-07 18:02:59 -07:00
Maximilian Hils
f62e976e1e py3++ 2016-07-07 17:29:22 -07:00
Maximilian Hils
64a867973d sni is now str, not bytes 2016-07-06 21:03:17 -07:00
Maximilian Hils
9c873d63f4 py3++, multidict fixes 
This commit improves Python 3 compatibility and fixes two multidict
issues:

 1. Headers.items(multi=True) now decodes fields
 2. MultiDict.clear(item) has been removed, as Python's
    MutableMapping already defines .clear() with different
    semantics. This is confusing for everyone who expects a
    dict-like object. `.pop("attr", None)` is not fantastic,
    but it's the Python way to do it.
 2016-07-06 19:53:29 -07:00
Maximilian Hils
55fae7cea9 Merge pull request #1319 from mitmproxy/fix-py27-encoding-issues 
Fix pathod log encoding
 2016-07-06 17:35:34 -07:00
Maximilian Hils
444f0a4c39 py3++ 2016-07-06 17:31:08 -07:00
Maximilian Hils
8b564bc934 fix pathod log encoding 2016-07-06 16:47:32 -07:00
Maximilian Hils
e84f5fcc72 Merge pull request #1315 from mitmproxy/better-unicode-handling 
remove clean_bin, clarify unicode handling
 2016-07-05 16:02:37 -07:00
Maximilian Hils
00a35aa895 Merge pull request #1297 from dufferzafar/py3-flow-export 
Python 3 - test_flow_export
 2016-07-05 15:17:18 -07:00
Maximilian Hils
d51cf543bb remove clean_bin, clarify unicode handling 2016-07-05 15:11:32 -07:00
Maximilian Hils
a6b3551934 raise ValueError if content-encoding is invalid 2016-07-04 13:58:09 -07:00
Thomas Kriechbaumer
23e295b37e py3: fix bytes vs. str 2016-07-03 23:22:04 +02:00
Shadab Zafar
d361177753 h2: move header parsing to netlib 2016-07-03 18:03:34 +05:30
Shadab Zafar
5b5a3ffa8e netlib: condition on PY2 rather than on PY3 
Let's just hope PY4 doesn't break this!
 2016-07-02 21:15:20 +05:30
Maximilian Hils
2f8a1fd2cb tests++ 2016-07-02 03:03:42 -07:00
Maximilian Hils
d9f797e7e6 make the linter happy 2016-07-02 02:11:00 -07:00
Maximilian Hils
6032c4f235 message.content -> .raw_content, implement .text 
This PR improves our handling of HTTP message body encodings:

- The unaltered message body is now accessible as `.raw_content`
- The "content-encoding"-decoded content (i.e. gzip removed) content
  is not `.content`, as this is what we want in 99% of the cases.
- `.text` now provides the "content-encoding"-decoded and then
  "content-type charset"-decoded message body.
- The decoded values for `.content` and `.text` are cached,
  so that repeated access and `x.text = x.text` is cheap.
- The `decoded()` decorator is now deprecated, as we can now just use
  `.content`. Similarly `HTTPMessage.get_decoded_content()` is
  deprecated.
 2016-07-02 01:51:47 -07:00
Maximilian Hils
2c09e0416b minor fixes 2016-07-01 17:49:18 -07:00
Maximilian Hils
fa72462798 fix tcp message handling 2016-07-01 17:17:16 -07:00
Maximilian Hils
536c7acd13 py3++ 2016-07-01 14:10:48 -07:00
Shadab Zafar
e0ed7699ca Handle case when scheme is None 2016-07-01 10:23:04 -07:00
Thomas Kriechbaumer
5d0de16f57 http2: improve error handling in tests 2016-06-22 22:09:57 +02:00
Thomas Kriechbaumer
eb3ed87100 move custom HTTP/2 stack from netlib to pathod 2016-06-17 15:06:01 +02:00
Shadab Zafar
d9b940c21e Py3: websockets 2016-06-15 09:49:38 +02:00
Shadab Zafar
0f1aa2b78e Py3: fix http2 bytes issue 2016-06-15 09:49:38 +02:00
Aldo Cortesi
a63817f2b8 Merge pull request #1253 from mhils/fix-debug 
Fix Debug Mode on Windows
 2016-06-14 17:12:02 +12:00
Aldo Cortesi
e6fd98bb72 Start rolling out leak detection in netlib 2016-06-14 14:32:18 +12:00
Maximilian Hils
3c9a70b761 fix debug mode on Windows 2016-06-13 18:06:44 -07:00
Aldo Cortesi
e58a2bf095 Merge branch 'master' into appveyor 2016-06-12 11:27:16 +12:00
Aldo Cortesi
dc545ca0f6 Make the tcp connection closer cancellable 
And use this to make pathoc error handling more sophisticated
 2016-06-12 11:17:05 +12:00
Aldo Cortesi
728d5caa22 Fold mitmproxy.version and pathod.version into netlib.version 2016-06-12 10:29:12 +12:00
Aldo Cortesi
e9f8eb1f6b Satisfy lint 2016-06-11 23:13:41 +12:00
Aldo Cortesi
7b86560ded debug: On SIGUSR2, we dump tracebacks for all threads to screen 2016-06-11 23:07:42 +12:00
Aldo Cortesi
53b2fd545b Zap stray semicolon 2016-06-11 21:26:38 +12:00
Aldo Cortesi
5566a1f0e6 debug: num_fds is posix-only 2016-06-11 21:25:34 +12:00
Aldo Cortesi
0848d1085e debug: add OpenSSL 2016-06-11 21:06:41 +12:00
Aldo Cortesi
3f240b1875 debug: verison -> version 
@resam blew our sponsorship - t-mobile is up next
 2016-06-11 21:04:13 +12:00
Aldo Cortesi
09edbd9492 Improve debugging of thread and other leaks 
- Add basethread.BaseThread that all threads outside of test suites should use
- Add a signal handler to mitmproxy, mitmdump and mitmweb that dumps resource
information to screen when SIGUSR1 is received.
- Improve thread naming throughout to make thread dumps understandable
 2016-06-11 19:52:24 +12:00
Aldo Cortesi
5b9f07c81c debug.sysinfo: tests and coverage 2016-06-11 17:56:17 +12:00
Aldo Cortesi
e367b88195 Add a --sysinfo flag to all daemons 
This dumps all the platform information and mitmproxy version data we'd
normally need to troubleshoot an issue.
 2016-06-11 16:40:21 +12:00
Aldo Cortesi
c421c41307 Remove odict 
- Adds default implementations for _kconv and _reduce_values to MultiDict.
Without these, operations fail in really, really non-obvious ways.
- Replace the remaining few instances of ODict

Fixes #1159
 2016-06-09 13:28:43 +12:00
Maximilian Hils
c98c83190b tests++ 2016-06-07 01:03:08 -07:00
Aldo Cortesi
1ffc273c94 Utils cleanups 
- Move more stuff that belongs in netlib.human
- Move some stuff to near the only use
- Zap mitmproxy.utils.timestamp(). I see the rationale, but we used it
interchangeably with time.time() throughout the project. Since time.time()
dominates in the codebase and timestamp() is such low utility, away it goes.
 2016-06-07 17:12:52 +12:00
Maximilian Hils
38d2e6432c minor fixes 2016-06-06 19:10:19 -07:00
Maximilian Hils
7cb7d9ad32 netlib: always disconnect when using context manager 2016-06-06 15:10:03 -07:00
redfast00
a724fd606a
Now keeps header capitalization 2016-06-05 20:47:10 +02:00
Aldo Cortesi
c0c45c051a Fix test that may fail due to binary header data 
This is just inherently not a determinisitc test. We don't use the log HTTP
interface any more, so it can just go. A more radical "solution" is inbound
shortly.

Fixes #1207
 2016-06-05 11:47:52 +12:00
Aldo Cortesi
6943d7e397 More explicit name for the tcp.Server handler counter 2016-06-03 13:57:12 +12:00
Aldo Cortesi
e60860e65d Make tcp.Client.connect return a context manager that closes the connection 2016-06-03 12:06:36 +12:00
Aldo Cortesi
c5076f5e01 Implement a service connection handler counter, use it in Pathod test suite 
Lots of failures, but that's a good thing.
 2016-06-03 11:47:07 +12:00
Aldo Cortesi
09da1febbd Shift a bunch more string-related functions to strutils 2016-06-02 13:04:19 +12:00
Aldo Cortesi
cccdc98426 Utils reorganisation: add netlib.strutils 
Extract a number of string and format-related functions to netlib.strutils.
 2016-06-02 12:31:41 +12:00
Aldo Cortesi
eaa3b308f7 Fix non-deterministic test failures in export 
We had various places in the code where we relied on incidental order of dict
keys. Add a helper to multidict, and fix.
 2016-06-02 11:37:18 +12:00
Aldo Cortesi
92b24c6653 Merge pull request #1183 from mitmproxy/module-imports 
use module-level imports only
 2016-06-01 16:26:55 +12:00
Maximilian Hils
0a2e7a97cd use module-level imports only 2016-05-31 19:06:57 -07:00
Aldo Cortesi
44fdcb4b82 Reorganise netlib imports according to Google Style Guide 2016-06-01 11:12:10 +12:00
Maximilian Hils
30fff1fb32 be more strict about module namespaces 2016-05-31 14:05:57 -07:00
Aldo Cortesi
4da125b6a0 Fix tests harder 2016-05-31 20:51:06 +12:00
Aldo Cortesi
42e91fcfe1 http2.frame -> http2.framereader 2016-05-31 20:32:28 +12:00
Aldo Cortesi
40a030f215 Satisfy flake8 2016-05-31 19:58:28 +12:00
Aldo Cortesi
f62efed304 Unify and make symmetric pretty_size and parse_size 2016-05-31 19:45:48 +12:00
Aldo Cortesi
b2f63458fc Move human-friendly format functions to netlib.human, remove redundant implementations 2016-05-31 19:32:08 +12:00
Aldo Cortesi
ec34cae618 utils.multipartdecode -> http.multipart.decode 
also

utils.parse_content_type -> http.headers.parse_content_type
 2016-05-31 19:07:55 +12:00
Aldo Cortesi
15b2374ef9 netlib.utils.get_header_tokens -> netlib.http1.read.get_header_tokens 
Placing this next to its only use.
 2016-05-31 18:54:42 +12:00
Aldo Cortesi
da074bff01 isascii is unused 2016-05-31 18:51:40 +12:00
Aldo Cortesi
6dda2b2ee5 Module is part of the name - url.decode, not url.urldecode 
A pattern we need to use far more often in the codebase
 2016-05-31 18:46:19 +12:00
Aldo Cortesi
4e6c9c4e93 Extract url functions from netlib.utils and move to netlib.http.url 2016-05-31 18:42:56 +12:00
Aldo Cortesi
08fbe6f111 Start cleaning up netlib.utils 
- Remove http2 functions, move to http2.frame
- Remove Serializable, move to netlib.basetypes
 2016-05-31 17:16:31 +12:00
Aldo Cortesi
9ea68ebd28 Improve handling of pseudo-headers 
- The canonical source for :method, :scheme and :path are the .method, .scheme
and .path attributes on the request object.
- These pseudo-headers are stripped after reading the request, and re-inserted
just before sending.
- The :authority header remains, and should be handled analagously to the Host
header in HTTP1 with respect to display and user interaction.
 2016-05-31 14:36:17 +12:00
Aldo Cortesi
e8290a0eb4 Satisfy flake8 2016-05-30 12:41:44 +12:00
Thomas Kriechbaumer
e723c2d6ad fix bytes vs. str 2016-05-29 22:17:45 +02:00
Thomas Kriechbaumer
03167d5e71 fix headers order 2016-05-29 22:07:22 +02:00
Thomas Kriechbaumer
d76e5ad1ee raise a more verbose error 2016-05-29 19:28:59 +02:00
Thomas Kriechbaumer
e2447406cc more style cleanup 2016-05-29 13:43:20 +02:00
Thomas Kriechbaumer
8eeab37a07 more style cleanup 
Use this to check:
flake8 --count mitmproxy netlib pathod examples test
 2016-05-29 11:55:12 +02:00
Aldo Cortesi
f3bee6f245 Merge pull request #1172 from cortesi/solidcore 
First steps to solidifying the core
 2016-05-29 20:03:40 +12:00
Maximilian Hils
85aa5da6e3 fix tests 2016-05-28 19:31:43 -07:00
Maximilian Hils
b70663f434 minor improvements 2016-05-28 19:10:49 -07:00
Aldo Cortesi
ed415877d4 Merge branch 'master' into solidcore 2016-05-29 11:14:46 +12:00
Thomas Kriechbaumer
e5038c9ab7 netlib: fix most flake8 offenses 2016-05-28 22:17:02 +02:00