Grasscutters/mitmproxy
3
0
Fork 0
mirror of https://github.com/Grasscutters/mitmproxy.git synced 2024-12-04 20:47:25 +00:00
Code Issues Projects Releases Wiki Activity
7,337 Commits 1 Branch 67 Tags 50 MiB
69726f180a
Commit Graph

1762 Commits

Author SHA1 Message Date
lymanZerga11
26a17a3d82 Update test_flowlist.py 2017-02-13 15:05:29 +08:00
lymanZerga11
a912d67c06 Update test_flowlist.py 2017-02-13 10:47:50 +08:00
lymanZerga11
577fb818b9 Update test_flowlist.py 2017-02-13 10:39:48 +08:00
lymanZerga11
f5b30b8872 Update test_flowlist.py 2017-02-12 23:36:26 +08:00
lymanZerga11
df8a5aa9be Update test_flowlist.py 2017-02-12 23:23:23 +08:00
lymanZerga11
c622e4a649 Create test_flowlist.py 2017-02-12 23:10:49 +08:00
Maximilian Hils
3067a971f9 Merge pull request #2000 from s4chin/add-jpeg-parser 
Add jpeg parser
 2017-02-11 23:00:14 +01:00
Sachin Kelkar
245e24dcf3 Add sources of images, remove copyrighted image 2017-02-12 01:19:36 +05:30
Maximilian Hils
71742654e3 Merge pull request #2005 from ujjwal96/script-reload 
Closes #1828 script reloads on py file changes
 2017-02-11 01:45:14 +01:00
Thomas Kriechbaumer
c8c79cc291 coverage++ 2017-02-10 22:55:15 +01:00
Sachin Kelkar
8eb1d34644 Add app1 marker and tests 2017-02-10 22:40:19 +05:30
Ujjwal Verma
809207195d closes #1828 script reloads on py file changes 2017-02-10 22:21:12 +05:30
Sachin Kelkar
c4e141a000 Add jpeg app0 sof0 com parsing 2017-02-10 11:16:23 +05:30
Ujjwal Verma
d6465b907f Closes #1580 Gives warning when explicit proxy configured in transparent mode (#1996) 2017-02-09 20:56:06 +01:00
Maximilian Hils
380ff50e57 Merge pull request #2003 from Kriechi/coverage++ 
test refactoring and coverage++
 2017-02-09 17:45:50 +01:00
Maximilian Hils
da8444b11f flow_count() -> len(flows) 2017-02-09 16:44:27 +01:00
Maximilian Hils
1084588103 Merge pull request #1998 from mhils/fix-change-upstream 
Fix change_upstream_proxy_server
 2017-02-09 16:39:20 +01:00
Thomas Kriechbaumer
d10560d54c improve coverage report 2017-02-09 16:08:36 +01:00
Thomas Kriechbaumer
2ff5d72236 minor changes 2017-02-09 16:08:36 +01:00
Thomas Kriechbaumer
0299bb5b2e eventsequence: coverage++ 2017-02-09 16:08:36 +01:00
Thomas Kriechbaumer
5a3976c43e coverage whitelist -> blacklist 2017-02-09 16:08:36 +01:00
Thomas Kriechbaumer
f6cea09d5a stateobject: coverage++ 2017-02-09 16:08:36 +01:00
Thomas Kriechbaumer
1847cf175c websockets, tcp, version: coverage++ 2017-02-09 16:08:35 +01:00
Thomas Kriechbaumer
ae94ca6fa9 remove deprecated flow_count function 2017-02-09 16:08:35 +01:00
Maximilian Hils
9b97b63891 fix change_upstream_proxy_server 2017-02-09 15:24:31 +01:00
Maximilian Hils
7a205e80aa improve stateobject tests 2017-02-09 14:21:25 +01:00
Thomas Kriechbaumer
7a9d40817c pytest.raises: shim new API 2017-02-08 20:04:07 +01:00
Maximilian Hils
28c0596742 Merge pull request #1997 from s4chin/add-gif-parser 
Add gif parser and tests
 2017-02-08 12:09:48 +01:00
Sachin Kelkar
66ad95c330 Fix gif parsing error on some gifs and add test 2017-02-08 00:03:07 +05:30
Sachin Kelkar
4ef5de2cce Add gif parser and tests 2017-02-06 17:57:59 +05:30
Ujjwal Verma
c622622c59 Encoding fixes and tests 2017-02-05 18:59:01 +05:30
Thomas Kriechbaumer
288f9a3857 fix flaky test 2017-02-05 10:42:47 +01:00
Thomas Kriechbaumer
966ffaa3d6 coverage++ 2017-02-04 13:48:55 +01:00
Maximilian Hils
53f298ac41 Merge pull request #1967 from s4chin/remove-pillow 
Add png parser
 2017-02-03 18:58:34 +01:00
Maximilian Hils
bbdb7300fd Merge pull request #1983 from Kriechi/coverage++ 
coverage++
 2017-02-03 18:56:56 +01:00
Sachin Kelkar
15548ff433 Feedback 2017-02-03 18:40:50 +05:30
Thomas Kriechbaumer
bcaaa2f40b coverage++ 2017-02-03 11:55:33 +01:00
Sachin Kelkar
0674485e76 Fix and cleanup 2017-02-03 15:02:35 +05:30
Sachin Kelkar
88dbfd5257 Add iTXt and zTXt tests 2017-02-03 14:12:54 +05:30
Sachin Kelkar
abef020e07 Fix as per feedback and add more tests 2017-02-03 14:12:54 +05:30
Sachin Kelkar
6202958048 Add png parser 2017-02-03 14:12:54 +05:30
Ujjwal Verma
e10c36fe11 Added test 2017-02-03 02:58:02 +05:30
Thomas Kriechbaumer
79aa994275 fix merge 2017-02-02 17:26:31 +01:00
Thomas Kriechbaumer
4f0b2bc4de adapt coverage checks 2017-02-02 14:20:56 +01:00
Thomas Kriechbaumer
ae008ed80b replace tutils.raises with pytest.raises + shim 2017-02-02 12:59:01 +01:00
Thomas Kriechbaumer
ec92d7f67e cleanup test utils 2017-02-02 12:59:01 +01:00
Thomas Kriechbaumer
6e329595ca add test coverage protection 2017-02-02 11:39:33 +01:00
Thomas Kriechbaumer
380d8ec370 increase test coverage 2017-02-02 10:15:01 +01:00
Thomas Kriechbaumer
3ae060f0d3 fix console test 2017-02-02 10:13:12 +01:00
Maximilian Hils
7b2a986cea tests++ (#1976) 2017-02-01 17:52:01 +01:00
lymanZerga11
4b10212caf Add except clause to catch script parsing errors (#1929) 2017-02-01 15:27:10 +01:00
Maximilian Hils
212d9f1b98 Merge pull request #1948 from amm0nite/fix_dns_spoofing_example 
Fix for dns_spoofing.py example
 2017-02-01 15:25:51 +01:00
Aldo Cortesi
914659e888 Regularise setheaders options 
As per replacements:

- Make the option type a string/tuple union
- Localise parsing strictly within the addon
- Adapt the console editor (we'll find a more elegant solution later)
 2017-02-01 12:22:05 +13:00
Ammonite
9e3f06b7f2 Better test 2017-01-31 23:23:13 +01:00
Aldo Cortesi
4cc75a9560 Revamp replacement hooks 
- Replacement specifiers can be either strings or tuples. This lets us cope
gracefully with command-line parsing (and posible quick interactive
specification) without having to special-case replacement hooks, or have
knowledge of hook specification leak outside the addon. We can also now use the
same command-line spec format in config files.
- Split replacement and replacement from file into separate addons and options.
Files are now read on each replacement, so you can edit replacement files in
place without restart.
- Modernise the test suite to use addon test helpers.

TODO: editing and displaying replace-from-file in console app
 2017-02-01 11:10:28 +13:00
Aldo Cortesi
02f51d043d Merge pull request #1965 from cortesi/optname 
Consolidate option names
 2017-02-01 08:39:35 +13:00
Thomas Kriechbaumer
ff379b7665 fully test addonmanager (#1970) 2017-01-31 10:57:22 +01:00
Ammonite
e2e15df358 Add a test for the host replacement 2017-01-30 23:44:13 +01:00
Aldo Cortesi
35aff3b783 Consolidate option names 
- Onboarding app options to onboarding*
- Console-specific options to console_*
- Web-specific options to web_*
 2017-01-30 11:15:12 +13:00
Aldo Cortesi
356c4987a2 addons: unit tests to 100% 
This patch pushes the coverage of ./tests/mitproxy/addons to 100% of the addons
module.
 2017-01-29 11:44:49 +13:00
Thomas Kriechbaumer
d24f76c98e fix newlines on windows 2017-01-25 18:51:09 +01:00
Thomas Kriechbaumer
5549757268 fix alpn requirement in web tests 2017-01-25 18:18:18 +01:00
Thomas Kriechbaumer
3028e06fd2 simplify termlog outfile handling 2017-01-25 17:50:07 +01:00
nish21
ab45e4d183 write errors to stderr (#1952) 2017-01-25 17:49:56 +01:00
Maximilian Hils
2eaac31344 Merge pull request #1945 from Kriechi/ca-expired 
fix #939
 2017-01-23 21:22:38 +01:00
Thomas Kriechbaumer
5792e2c483 test disable_h2c_upgrade addon 2017-01-23 18:38:27 +01:00
Thomas Kriechbaumer
d674de298c test websocket addon 2017-01-23 15:45:53 +01:00
Thomas Kriechbaumer
006eb39cc5 fix #939 2017-01-23 15:14:56 +01:00
Thomas Kriechbaumer
127c69c3ac nuke mock dependency 
This is already included in Python 3.3+ as unittest.mock
 2017-01-22 14:11:28 +01:00
Thomas Kriechbaumer
72b753c60f provide git information with --version 
fixes #1848
 2017-01-19 20:27:29 +01:00
Maximilian Hils
88f3459c7d stronger tests, lint 2017-01-07 23:08:50 +01:00
Maximilian Hils
b1ec7e78cd Revert "add strutils.replace_surrogates" 
This reverts commit 15ae88db08dbf2725c8c9f945f2006c4b2a10d46.
 2017-01-07 23:08:50 +01:00
Maximilian Hils
c21ee90deb add strutils.replace_surrogates 2017-01-07 23:08:50 +01:00
Maximilian Hils
042261266f minor encoding fixes 
- native() -> always_str()
  The old function name does not make sense on Python 3 only.
- Inline utility functions in message.py.
 2017-01-07 23:08:50 +01:00
lymanZerga11
3ea76a7f3e Controls the length of URL displayed using the -d option (#1903) 2017-01-05 14:37:20 +01:00
chhsiao90
a5f1215eb2 Allow underscore in hostname 2016-12-30 21:03:22 +08:00
Maximilian Hils
eab360a02b fix IO type checking 2016-12-28 14:21:19 +01:00
Maximilian Hils
0929e74b4e fix compat with Python 3.5.0 2016-12-28 14:20:53 +01:00
Maximilian Hils
c78ffbf16d fix url.parse tests for Python 3.6 
This is a simpler version of @Kriechi's patch.
 2016-12-27 19:09:56 +01:00
Thomas Kriechbaumer
24751965f9 remove output of dumper during tests 2016-12-21 23:22:14 +01:00
Thomas Kriechbaumer
a196493a7a remove output of webmaster during tests 2016-12-21 23:13:53 +01:00
Maximilian Hils
77cd9224f9 Merge pull request #1880 from mhils/1877 
Fix #1877
 2016-12-19 19:04:45 +01:00
Maximilian Hils
4f3b50e417 fix io compat with previous releases 2016-12-19 16:19:37 +01:00
Maximilian Hils
fbce37054f fix #1877 2016-12-19 16:10:41 +01:00
Maximilian Hils
c4e9000021 fix #1858 2016-12-19 01:15:10 +01:00
Aldo Cortesi
cf15a3c3ef Merge pull request #1862 from mhils/remove-intercept-events 
Remove intercept events, update view manually
 2016-12-17 09:15:11 +13:00
Maximilian Hils
388fa7e716 remove intercept events, update view manually 2016-12-16 05:43:38 +01:00
Aldo Cortesi
e387c68b38 console: mark view indiciator, fix empty flow list behaviour 
Fixes #1782
 2016-12-16 11:57:45 +13:00
Aldo Cortesi
6b5673e849 Merge pull request #1845 from mhils/mitmweb-improvements 
Mitmweb Improvements
 2016-12-16 10:04:12 +13:00
Maximilian Hils
727abdba44 Merge pull request #1852 from mhils/simplenamespace 
Use types.SimpleNamespace instead of custom NS class
 2016-12-13 15:06:14 +01:00
Maximilian Hils
44f94c8844 use types.SimpleNamespace instead of custom NS class 2016-12-13 14:23:21 +01:00
Maximilian Hils
e5b3c8bed3 fix #1850 2016-12-13 14:12:18 +01:00
Maximilian Hils
d854e08653 [web] various fixes 2016-12-12 00:08:29 +01:00
Aldo Cortesi
a99bf0814c console: clean path prompt handling 
- Don't trigger callback if no path is specified
- Expand userdir centrally
- Catch IO errors from callbacks and turn them into status messages
 2016-12-12 09:13:43 +13:00
Aldo Cortesi
ca33bea296 Extend type checker validate Sequence specs 2016-12-12 07:39:59 +13:00
Maximilian Hils
3e37cbd061 minor fixes 2016-12-10 12:06:33 +01:00
Maximilian Hils
123ef043dc add new xml/html pretty-printer 🎉 2016-12-10 11:40:07 +01:00
Maximilian Hils
293b79af91 remove lxml-dependent code 2016-12-10 10:19:05 +01:00
Maximilian Hils
a7ba2f7b46 update tests, increase coverage, add type info 2016-12-09 21:26:02 +01:00
Aldo Cortesi
e943147fc3 options: add a merge method to assimilate options 
Fixes #953
 2016-12-08 16:12:40 +13:00
Aldo Cortesi
b2695dbc6a Minor tweaks 2016-12-08 10:21:06 +13:00
Aldo Cortesi
a617e3b5f7 Amalgamate all the Options objects 2016-12-08 10:21:06 +13:00
Aldo Cortesi
d742d4fb8c Replace configargparse with argparse, enable new options loading 
This commit ditches configargparse and enables the new config file format. The
default location is ~/.mitmproxy/config.yaml. Unifying all of the various
Options objects will follow in the next patch.
 2016-12-08 10:21:06 +13:00
Aldo Cortesi
150372e297 options: defaults are a class attribute 2016-12-08 10:21:06 +13:00
Aldo Cortesi
00c897a185 options: save defaults, add .reset() to restore defaults 
Use .reset() in console app to clear options.
 2016-12-08 10:21:06 +13:00
Aldo Cortesi
d74cac265a Add YAML serialization of options 
This uses ruamel.yaml. The library seems well-supported, and can do in-place
modification of config files that retains user comments and file structure.
 2016-12-08 10:21:06 +13:00
Aldo Cortesi
c94cd512d1 options: defaults are a class attribute 2016-12-08 10:21:06 +13:00
Aldo Cortesi
c062e302e9 Add OptManager.subscribe, use it to clean up palettes in console 
.subscribe lets you subscribe a function to a specified set of options.
 2016-12-08 10:21:06 +13:00
Aldo Cortesi
0c091bd92b Options - avoid mutation, API cleanup, has_changed 
- Always return a deepcopy of options to avoid accidental mutation of options
state.
- Remove .get(opt, default). This is an inappropriate API for Options - trying
to retrieve an option that doesn't exist should always be an error.
- Add the has_changed method that checks if an option differs from the default,
use it in mitmproxy console.
 2016-12-08 10:21:06 +13:00
Aldo Cortesi
b231836c70 Get rid of tfile testing option 
It's weird, it's ugly, it's getting in the way of my options refactoring, and
it must therefore die.
 2016-12-08 10:21:05 +13:00
Aldo Cortesi
297493801d Clean up dump tests 
- Remove tests that redundantly test addon funtionality that we've already
tested more comprehensively elsewhere.
- Extend to 100% coverage for tools/dump.py
 2016-12-08 10:21:05 +13:00
Aldo Cortesi
b4d33aaebf options: save defaults, add .reset() to restore defaults 
Use .reset() in console app to clear options.
 2016-12-08 10:21:05 +13:00
Thomas Kriechbaumer
47196e8676 tls_parser: allow optional extensions 
fixes #1816
 2016-12-06 19:51:57 +01:00
Thomas Kriechbaumer
1a36efbb6a simplify ALPN and OpenSSL on macOS 2016-12-04 11:04:06 +01:00
Thomas Kriechbaumer
cc6aa1f542 websocket: update close handshake tests 2016-11-29 23:47:19 +01:00
Maximilian Hils
34bf3a2496 fix #1783 (#1794) 2016-11-28 13:29:06 +01:00
Maximilian Hils
3154dc87fd fix pyinstaller 
amongst other things, fix #1746
 2016-11-28 00:51:52 +01:00
Maximilian Hils
568f40c810 Fix transparent mode initialisation, refactor mitmproxy.platform (#1787) 2016-11-25 17:32:23 +01:00
Maximilian Hils
222106916e Merge branch 'mitmweb-eventlog' 2016-11-23 22:47:50 +01:00
Maximilian Hils
45332006a3 mitmweb: 100% app test coverage, numerous fixes 2016-11-23 22:35:07 +01:00
Thomas Kriechbaumer
d3bd04dec0 disable TCP support for now 2016-11-23 10:18:45 +01:00
Thomas Kriechbaumer
5dfc199086 websocket: add tests 2016-11-23 10:18:45 +01:00
Thomas Kriechbaumer
4beb693c9c websocket: change cmdline option 2016-11-23 10:18:45 +01:00
Thomas Kriechbaumer
ffb3988dc9 rename WebSocket{s,} protocol 2016-11-23 10:18:45 +01:00
Maximilian Hils
f1662cbfd7 add eventstore tests 2016-11-23 01:27:51 +01:00
Yoginski
ebff5f2466 Descriptive error message for SSL context initialization failure (#1767) 2016-11-21 09:40:09 +01:00
Maximilian Hils
9af8f4bb31 organize examples 
This commit is largely based on work by Thiago Arrais (@thiagoarrais)
and Shane Bradfield (@l33tLumberjack). I wasn't really able to get their
PR reasonably merged onto the latest master, so I reapplied their changes
manually here and did some further improvements on that.
 2016-11-21 02:28:10 +01:00
Maximilian Hils
f74e561524 rewrite absolute-form HTTP requests to relative form, fix #1759 (#1765) 2016-11-20 17:21:21 +01:00
Thomas Kriechbaumer
f45034e8f1 tests: fix race condition 2016-11-20 17:02:26 +01:00
Maximilian Hils
5ce370e2a9 lint! 2016-11-18 12:59:59 +01:00
Aldo Cortesi
8065b44eed addons.view: clarify modification events 
This adds a set of store modification events, and uses them for flow settings.
This addresses a bug where settings could persist even after flows were deleted.
 2016-11-15 09:53:17 +13:00
Aldo Cortesi
9b08279c7c addons.proxyauth: out with the old, in with the new 
- Strip out old auth mechanisms, and enable addon
- Disable web app auth for now - this should just use the Tornado auth stuff
 2016-11-13 19:59:59 +13:00
Aldo Cortesi
dc88b7d110 addons.proxyauth: complete and test 2016-11-13 18:14:23 +13:00
Aldo Cortesi
e644d2167c stub out proxyauth addon 
Stub out basic workings, add and test configure event.
 2016-11-13 16:22:33 +13:00
Aldo Cortesi
fe01b1435a upstream_proxy_auth -> upstream_auth 
Also clarify what this does in commandline help.
 2016-11-13 11:50:28 +13:00
Aldo Cortesi
3b00bc339d Complete upstream authentication module 
- Handles upstream CONNECT and regular requests, plus HTTP Basic for reverse
proxy
- Add some tests to make sure we can rely on the .via attribute on server
connections.
 2016-11-13 11:43:27 +13:00
Aldo Cortesi
a9b4560187 Refine handling of HTTP CONNECT 
- CONNECT requests do not generate the usual http events. Instead, they
generate the http_connect event and handlers then have the option of setting an
error response to abort the connect.
- The connect handler is called for both upstream proxy and regular proxy CONNECTs.
 2016-11-12 18:28:37 +13:00
Aldo Cortesi
38f8d9e541 Add the http_connect event for HTTP CONNECT requests 2016-11-12 12:44:43 +13:00
Aldo Cortesi
bc01a146b0 Upstream proxy auth to addon 2016-11-12 11:58:04 +13:00
Aldo Cortesi
3958940420 Test failure during 100-continue 
Also:

- Remove duplicate and unused code
- Tighten scope of HttpReadDisconnect handler - we only want to ignore this for
the initial read, not for the entire block that includes things like the expect
handling.
 2016-11-12 11:58:04 +13:00
Aldo Cortesi
82ac7d05a6 Bug: ask requestheaders before request body is read 
Also add the beginnings of a test suite to exercise issues like this.
 2016-11-12 11:58:04 +13:00
Thomas Kriechbaumer
184e29e119 http2: add push-promise metadata test 2016-11-05 05:13:36 +01:00
Aldo Cortesi
067198a5dd Remove deprecated interfaces 2016-11-04 11:47:47 +13:00
Aldo Cortesi
6c8c4465d9 script.concurrent: tests to taddons 2016-11-04 10:59:41 +13:00
Aldo Cortesi
4f4db223fe addons: convert last of the test suite to taddons 2016-11-04 09:17:56 +13:00
Aldo Cortesi
deb66d3cac addons.setheaders: fix configure bug, tests to taddons 2016-11-04 09:09:39 +13:00
Aldo Cortesi
b51a96081a addons.view: test suite to 100% 
This takes coverage for all of /addons to 100%
 2016-11-04 08:55:01 +13:00
Aldo Cortesi
17e828b243 addons.view: test improvements 2016-11-03 20:53:06 +13:00
Aldo Cortesi
5ade93f2ad addons.streambodies: 100% test coverage 2016-11-03 20:16:07 +13:00
Aldo Cortesi
d7d6edb3d1 addons.serverplayback: 100% test coverage 2016-11-03 20:05:21 +13:00
Aldo Cortesi
e1fc80937d addons.serverplayback: test suite to taddons 2016-11-03 19:43:33 +13:00
Aldo Cortesi
e9a96f4d7f addons.script: 100% test coverage 2016-11-03 19:36:34 +13:00
Aldo Cortesi
d9538637c3 addons.script: convert to test.taddons 2016-11-03 18:57:34 +13:00
Aldo Cortesi
d31f2698a5 addons.onboarding: remove duplicate code 2016-11-03 16:20:38 +13:00
Aldo Cortesi
9fc6674151 addons.intercept: 100% test coverage 2016-11-03 16:16:29 +13:00
Aldo Cortesi
77e6dfe35c streamfile: make options less crazy 
Instead of a (file, mode) tuple, move to "streamfile" as the path, and
"streamfile_append" to specify if we should append or not.

Leave the command-line options intact for now.
 2016-11-03 15:18:55 +13:00
Aldo Cortesi
9f77c79227 FileStreamer -> StreamFile 
options.wfile -> options.streamfile
 2016-11-03 15:00:18 +13:00
Aldo Cortesi
43a8221989 addons: filestreamer test cov to 100% 
Also address a bug in taddons that caused options changes that raised errors to
be retained rather than rolled back.
 2016-11-03 14:50:23 +13:00
Aldo Cortesi
fbdce4b629 addons filstreamer: tests and bugfixes 
- Move tests to taddons
- Fix a bug where options updates caused streaming to halt
 2016-11-03 12:44:19 +13:00
Aldo Cortesi
5109fd8ecb Change antiquated test cipher suites 
This breaks Travis, possibly due to OpenSSL version.
 2016-11-03 08:23:19 +13:00
Aldo Cortesi
b867fb35a3 addons: dumper spit and polish 
- 100% test coverage
- Cleanups
- Add test/mitmproxy/addons/dumperview.py, a small utility for viewing dumper
output variations
 2016-11-02 22:47:49 +13:00
Aldo Cortesi
a75b3474a4 tests: convert test_dumper to use taddons 2016-11-02 11:17:08 +13:00
Aldo Cortesi
59b269425f tests: client_playback coverage 2016-11-02 10:59:33 +13:00
Aldo Cortesi
7e6d014f8f tests: pathod/tutils.py -> pathod/tservers.py 
And remove all aliases for mitmproxy.test.tutils
 2016-11-02 10:06:25 +13:00
Aldo Cortesi
c55e8d8f62 tests: clean up test/mitmproxy/tutils.py 
Remove aliases, use mitmproxy/test/tutils instead.
 2016-11-02 09:44:18 +13:00
Aldo Cortesi
fc7606bd98 tests: minor cleanup for tests/mitmproxy/tutils.py 2016-11-02 08:51:26 +13:00
Aldo Cortesi
83fdd82a52 addons anticom, anticache: expand tests, new truss 2016-11-01 17:38:45 +13:00
Aldo Cortesi
50deaf56bf Remove addons.State 
- Do a first-pass port of mitmweb. Some stuff still broken.
- Remove more all State tests from the test suite
 2016-11-01 10:06:28 +13:00
Aldo Cortesi
9d9735dd07 Replace addons.state in test suite 
Use a super-simple flow recorder instead.
 2016-11-01 09:36:24 +13:00
Aldo Cortesi
daf355bb4c console: add caching sort keys 
This is a tad complicated. The underlying sorted list implementation expects
the sort key to be stable for the lifetime of the object. However, if we sort
by size, for instance, the sort order changes as the flow progresses through
its lifecycle. We address this through two means:

- Let order keys cache the sort value by flow ID.
- Add a facility to refresh items in the list by removing and re-adding them
when they are updated.
 2016-10-30 21:11:30 +13:00
Aldo Cortesi
9abfb1aac2 console: re-add follow focus 2016-10-30 19:47:32 +13:00
Aldo Cortesi
f08b57fb9b console: "v" to reverse flow order 2016-10-30 17:57:43 +13:00
Aldo Cortesi
a67a591893 console: o for flow order 
"o" shortcut and --order cmdline option to change flow order.

Options is now bound to "O".
 2016-10-30 17:42:34 +13:00
Aldo Cortesi
6b6c44551a addons.view: Add first-class support for marking 
Marking now works differently - it's no longer just another filter, it's
applied in addition to the filter. This means you can apply a filter, mark some
flows, and then toggle between marked and unmarked flows matching the filter.

I'm leaving the filter for marked flows intact - it will come in handy in other
situations.
 2016-10-30 15:45:36 +13:00
Aldo Cortesi
2b76db1272 console: Port from state to view 
First phase of the port - basic flow list and flow view functionality working.
More to come.
 2016-10-30 14:56:36 +13:00
Aldo Cortesi
b9eb1a3479 Add options.default_contentview 
This is the first step in a longer process of revamping content views. For the
moment, the option is not exposed on the command line.
 2016-10-30 12:05:37 +13:00
Aldo Cortesi
944dcbaaa0 Cleanups: remove intercept from addons.state, unused imports 2016-10-30 11:13:05 +13:00
Aldo Cortesi
81a00f6f76 Replace interception in mitmproxy and mitmweb with addon 
Fixes #1674
 2016-10-30 11:07:09 +13:00
Aldo Cortesi
d1f14961ed test.taddon: Add cycle() method, use it to test addons.intercept 2016-10-30 10:52:57 +13:00
Aldo Cortesi
b4904d33ba Move test flow creators to mitmproxy.test.tflow 2016-10-30 10:12:39 +13:00
Aldo Cortesi
2dc3284fbb Add addons.intercept 
- Add an addon to handle intercept based on a filter pattern
- Start sketching out a nicer testing truss for addons in mitmproxy.test.taddon
 2016-10-29 15:44:48 +13:00
Aldo Cortesi
71d2636594 Sync 2016-10-29 12:35:10 +13:00
Aldo Cortesi
005c22445b addons.view: focus.focusflow -> focus.flow 2016-10-29 11:55:44 +13:00
Aldo Cortesi
7ecaeb0214 addons.view.focus: next and prev methods 2016-10-29 11:50:10 +13:00
Aldo Cortesi
32a0a7b860 addons.view: flow settings 
Add a flow settings mechanism, enable focus and settings unilaterally.
 2016-10-29 11:37:32 +13:00
Aldo Cortesi
14df969434 addons.view.focus: Better handling of view refresh 
When we refresh and our current focus goes out of scope, we set the focus to
the element nearest the old focus.
 2016-10-29 11:08:35 +13:00
Aldo Cortesi
90e7142b5c addons.View: better order reversal 
Deal with some subtleties in order reversal, add a toggle method that emits
refresh.
 2016-10-29 10:55:57 +13:00
Aldo Cortesi
12a70d03ad addons.view: Add a focus tracker 2016-10-29 10:17:29 +13:00
Aldo Cortesi
9dcc3a3e20 addons.View: hook up signals 2016-10-29 08:25:19 +13:00
Aldo Cortesi
69bacee1d8 Sketch out addons.View 
The first iteration of a replacement for addons.State
 2016-10-29 08:25:19 +13:00
Thomas Kriechbaumer
44ac370f08 upgrade h2 to >=2.5.0 2016-10-27 15:10:53 -07:00
Maximilian Hils
97f1236c99 Merge pull request #1670 from chhsiao90/display-cipher-#582 
Resolved #582: display ClientConnection select cipher of TLS
 2016-10-27 11:38:22 -07:00
chhsiao90
d52f35428c Show tls version on console 2016-10-26 14:36:14 +08:00
Maximilian Hils
b1bdae3d1c typecheck options, fix current inconsistencies 2016-10-25 20:45:48 -07:00
chhsiao90
960f2e8bf0 Resolved #582: display ClientConnection select cipher of TLS 2016-10-26 11:32:42 +08:00
Maximilian Hils
145c2892f7 Merge pull request #1664 from chhsiao90/sni-display-#1639 
Resolved #1639: display sni on ClientConnection
 2016-10-24 19:56:05 -07:00
chhsiao90
39ac29e37c Resolved #1639: display sni on ClientConnection 2016-10-25 10:46:53 +08:00
Maximilian Hils
ef4e9b2b85 Merge pull request #1656 from mhils/improve-export-2 
Improve Flow Export
 2016-10-24 19:19:58 -07:00
Sahil Chelaramani
7b3505336a replace mitmproxy.utils.lrucache with functools.lru_cache 2016-10-24 16:03:16 -07:00
Maximilian Hils
e87daa70f3 improve flow export 2016-10-22 20:32:39 -07:00
Maximilian Hils
ea2d6474bf Merge pull request #1649 from cortesi/kill 
Don't call error for flow kill
 2016-10-22 19:22:50 -07:00
Maximilian Hils
a1a792aeac various encoding fixes, fix #1650 2016-10-22 18:47:12 -07:00
Aldo Cortesi
6170493615 Don't call error for flow kill 
This is now the error handler on master, so whatever the intetion was it's now
definitely wrong.
 2016-10-22 15:56:42 +13:00
Aldo Cortesi
cc8b422d9d multidict: remove to_dict 
We never use it, and it is dangerously ambiguous when a key is associated with
a list.
 2016-10-21 11:42:21 +13:00
Aldo Cortesi
18ee6255c0 multidict: ditch ImmutableMultiDict 
A contorted class we only use for cookie attributes. We don't need it.
 2016-10-21 11:40:05 +13:00
Aldo Cortesi
8430f857b5 The final piece: netlib -> mitproxy.net 2016-10-20 11:56:38 +13:00
Aldo Cortesi
853e03a5e7 netlib.tutils -> mitmproxy.test.tutils 
There's a LOT more to be done refactoring our different conflicting test utils.
 2016-10-20 11:42:55 +13:00
Aldo Cortesi
01a449b5cb netlib.exceptions.* -> mitmproxy.exceptions 2016-10-20 11:27:05 +13:00
Aldo Cortesi
301d52d9d0 netlib.encoding -> netlib.http.encoding 
Encoding is highly specific to http, and only used within this module.
 2016-10-20 11:06:57 +13:00
Aldo Cortesi
f964d49853 netlib.certutils -> mitmproxy.certs 2016-10-20 11:02:52 +13:00
Aldo Cortesi
9870844b38 netlib.utils -> netlib.check 
Now only contains is_valid_[host,port]

Intermediate step - this will be in mitproxy.net soon.
 2016-10-20 10:46:47 +13:00
Aldo Cortesi
e0f3cce14c netlib.utils.[get,set]bit -> mitmproxy.utils.bits 2016-10-20 10:38:57 +13:00
Aldo Cortesi
9555126585 netlib.utils.BiDi -> mitmproxy.types.bidi.BiDi 2016-10-20 10:32:09 +13:00
Aldo Cortesi
a684585e7c netlib.debug -> mitmproxy.utils.debug 2016-10-20 10:25:36 +13:00
Aldo Cortesi
1ecb25cdc1 mitmproxy.types.[basethread,multidict,serializable] 2016-10-20 10:22:23 +13:00
Aldo Cortesi
f45f4e677e netlib.strutils -> mitmproxy.utils.strutils 2016-10-20 10:11:58 +13:00
Aldo Cortesi
1407830280 netlib.human -> mitmproxy.utils.human 2016-10-20 09:45:18 +13:00
Aldo Cortesi
069119364d Create mitmproxy.utils hierarchy 
- Add mitproxy.utils.lrucache, mitproxy.utils.data
 2016-10-20 09:35:55 +13:00
Aldo Cortesi
e73c7fe77e mitmproxy.protocol -> mitmproxy.proxy.protocol 
The protocols here are compltely proxy-specific, are only used from within the
proxy module, and are not exposed to users.
 2016-10-19 23:11:56 +13:00
Aldo Cortesi
24cf8da27e Move all tools into mitmproxy.tools, move models/* to top level 
The primary motivation here (and for all the other moving around) is to present
a clean "front of house" to library users, and to migrate primary objects to
the top of the module hierarchy.
 2016-10-19 20:26:05 +13:00
Aldo Cortesi
5a68d21e8c Remove flow module entirely, move contents to top level 
mitmproxy.flow.io -> mitmproxy.io
mitmproxy.flow.export -> mitmproxy.export
 2016-10-19 15:08:35 +13:00
Aldo Cortesi
7c32d4ea2a flow.state -> addons.state 2016-10-19 14:48:42 +13:00
Aldo Cortesi
22eebfd574 addons.Addons -> addonmanager, builtins -> addons 2016-10-19 14:39:39 +13:00
Aldo Cortesi
966418725b controller.Log -> log.Log 2016-10-19 14:14:59 +13:00
Aldo Cortesi
bce387a5a0 Kill flow.master - create master.Master 
Also extract events into .events
 2016-10-19 13:22:50 +13:00
Aldo Cortesi
264a09e778 Let's dispense with this fiction that we support multiple servers 
It's simple enough to re-add if needed, and until then is just cruft.
 2016-10-19 12:37:32 +13:00
Aldo Cortesi
87629586ae web app cleanups: tests and examples 2016-10-19 11:48:51 +13:00
Aldo Cortesi
ceb8caee98 Web apps to addons 
This commit removes the app registry, adds a wsgiapp addon base, and ports the
onboarding app to it.
 2016-10-19 11:40:51 +13:00
Aldo Cortesi
03cb5bb325 No need for all builtins to be loaded for server tests 2016-10-18 22:30:21 +13:00
Aldo Cortesi
466f5e56a1 Convert flow.state to an addon 
A first step, and already many things are much nicer.
 2016-10-18 21:51:36 +13:00
Aldo Cortesi
d5056c5627 accept_addon -> resume, intercept/resume events 2016-10-18 12:02:35 +13:00
Aldo Cortesi
825b02d495 master.duplicate_flow -> state.duplicate_flow 
This is a pure operation on state, and doesn't belong on master.

Part of prep to move state to an addon.
 2016-10-18 11:29:51 +13:00
Aldo Cortesi
b25d94a6ac Response body streaming to addon 2016-10-18 10:42:05 +13:00
Maximilian Hils
beed406058 fix tests 2016-10-17 03:37:42 -07:00
Thomas Kriechbaumer
94c9dd66e6 remove empty lines at beginning of file 2016-10-17 10:32:12 +02:00
Maximilian Hils
758860531a fix socks5 mode with tls 2016-10-16 23:05:51 -07:00
Aldo Cortesi
c774a9fec9 python3: clean up super and __future__ 2016-10-17 18:03:07 +13:00
Aldo Cortesi
a647b30365 python3: clean up class brackets 2016-10-17 17:37:08 +13:00
Aldo Cortesi
fb22f2ff4f Zap object base class 2016-10-17 17:37:08 +13:00
Aldo Cortesi
ce98a9219e test & examples: zap six 2016-10-17 16:45:45 +13:00
Aldo Cortesi
9a0195bf64 scripts: keep scripts just after the ScriptLoader in addon chain 
We need scripts to run _before_ filestreamer, so we can't just add them to the
end of the chain. This patch also fixes an issue that could cause scripts to be
initialised un-necessarily if only the order of scripts in options changed.
 2016-10-16 20:26:06 +13:00
Aldo Cortesi
fb69c9c345 docs: overview, classes, arguments 2016-10-16 20:26:06 +13:00
Maximilian Hils
e9284ab7ee add regression test for #1605 2016-10-14 01:22:30 -07:00
Aldo Cortesi
89d36713e3 certutils: cap the cert store size at 100 by default 
This should be enough to give us reuse without growing infinitely. This is part
of fixing the memory situation in mitmdump.

TODO: There's an opportunity here for a better algorithm, that expires certs
based on least-recently-accessed time, rather than oldest generated time.
 2016-10-05 10:44:31 +11:00
Thomas Kriechbaumer
90898f9084 use flt instead of filt for flowfilter expressions 2016-10-04 14:51:04 +02:00
Aldo Cortesi
882c363ec5 debug: Hard exit after debug signal 2016-10-04 16:48:13 +11:00
Aldo Cortesi
7fe09f4d26 Consolidate logging 
- Make log event handler work
- Move terminal event log into an addon
- Clean various log related errors and duplications
 2016-10-04 13:00:15 +11:00
Aldo Cortesi
2e48b60ff1 Clarify channel interaction pattern 
Before this patch, it wasn't clear when the arguments to an event could be
modified, and when a new object should be returned. After this patch, the right
thing to do is to modify the arguments in 99% of cases. The only exception to
this is root_layer, for which we can find a better structure down the track.
 2016-10-04 08:13:56 +11:00
Aldo Cortesi
1ae73b9157 Add requestheaders event 
HTTP1 only for now, HTTP2 coming next.
 2016-10-04 08:13:56 +11:00
Thomas Kriechbaumer
36c04f1631 fix flowfilter.match args 2016-10-03 12:04:17 +02:00
Thomas Kriechbaumer
bb60b76af4 use flowfilter.match 2016-10-03 11:45:54 +02:00
Thomas Kriechbaumer
ba84248910 rename mitmproxy.filt -> mitmproxy.flowfilter 2016-10-03 11:45:54 +02:00
Maximilian Hils
611dad12a7 Revert "Add API to programmatically create new requests (#1534)" 
This reverts commit ccbdcd684b,
which yielded a test timeout.
 2016-10-02 23:08:51 -07:00
Tyler St. Onge
ccbdcd684b Add API to programmatically create new requests (#1534) 2016-10-01 13:06:59 +02:00
Aldo Cortesi
070aa27cf5 parse_set_cookie header returns an empty list if no cookies are found 
This matches parse_cookie, and is more idiomatic.
 2016-09-28 15:10:31 +10:00
Aldo Cortesi
b21f076cc8 Merge branch 'multi-cookie' of https://github.com/dufferzafar/mitmproxy 2016-09-28 12:44:40 +10:00
Shadab Zafar
5383c15ff2 Add tests for comma separated Set-Cookies 2016-09-27 21:28:54 +05:30
Ryan Laughlin
ad6d675b20 In reverse proxy mode, always send SNI 2016-09-27 10:49:04 -04:00
Shadab Zafar
e8ffc5167b Can't have , in the sticky cookie test 
Since we support comma separated cookies now
 2016-09-27 16:44:09 +05:30
Shadab Zafar
9ec0790271 Make needed changes in tests 2016-09-27 16:44:09 +05:30
Shadab Zafar
85bf5e18f6 Update cookie tests 
This is needed since _read_pairs now returns a list of cookies each of which
is a list of [name, value] tuples.
 2016-09-27 16:44:08 +05:30
Maximilian Hils
eeec17902f update dependencies, minor fixes (#1577) 2016-09-24 21:01:48 -07:00
Maximilian Hils
0a643ad20f remove option argument for addons.add (#1576) 2016-09-24 18:21:12 -07:00
Maximilian Hils
9142da1a7d Merge pull request #1566 from mhils/issue-1546 
add websocket on/off switch, improve logging (fix #1546, fix #1547)
 2016-09-22 03:06:39 -07:00
Maximilian Hils
77868434e7 fix tests 2016-09-22 02:10:02 -07:00
Maximilian Hils
a3329c5a80 scripts: cut traceback properly, fix #1467 2016-09-22 02:03:12 -07:00
Maximilian Hils
9e0b935fa2 Merge pull request #1564 from mhils/issue-1554 
Fix Response.make content-length header
 2016-09-21 21:49:14 -07:00
Maximilian Hils
0b532789a0 fix Response.make content-length header 2016-09-21 20:34:01 -07:00
Maximilian Hils
770936f1f9 raise TypeError on invalid header assignment, fix #1562 2016-09-21 19:21:32 -07:00
Aldo Cortesi
cfb943bfdd Redesign client replay 
- Move to an addon
- Use a much simpler synchronisation mechanism
 2016-09-11 20:00:34 +12:00
Aldo Cortesi
6b9221ffeb Add "run_once" support to the script addon, use it in mitmproxy 
Fixes #1418
 2016-09-11 13:22:01 +12:00
Aldo Cortesi
9b814ca067 options.replay_ignore* -> options.server_replay_ignore* 2016-09-10 10:21:52 +12:00
Aldo Cortesi
8677e91074 options.nopop -> options.server_replay_nopop 2016-09-10 10:09:07 +12:00
Aldo Cortesi
5ebe8c1bc8 options.rheaders -> options.server_replay_use_headers 2016-09-10 10:09:07 +12:00
Aldo Cortesi
b0213a2c0e options.kill -> options.replay_kill_extra 2016-09-10 10:09:07 +12:00
Aldo Cortesi
4ff8a72521 Roll out synchronisation for mitmproxy tests 
This extends some of the work I did for pathod and netlib to the mitmproxy test
suite. It also fixes what may be a leak in replays.

Failing on connection leak is disabled on Windows for the moment.

Fixes #1535
 2016-09-10 09:18:11 +12:00
Aldo Cortesi
ea49b8a2e2 Merge pull request #1532 from cortesi/playback 
Playback and fix construct breakage
 2016-09-07 12:59:11 +12:00
Schamper
5728a1c900 Make 'none' synonymous to 'identity' 2016-09-06 17:00:08 +02:00
Aldo Cortesi
9abdd3a8d9 Move server playback in to an addon 
- Move server playback into an addon
- Implement a better sync strategy to decide when to exit if keepserving is
off. We now wait for the final flow played back to no longer be live.
- Leave interactive server playback in mitmproxy console broken for now - there
are broader addon-related changes that need to be made for that, and this patch
is already big.

Fixes #1229
 2016-09-06 23:48:02 +12:00
Schamper
85e1539d0a Add test 2016-09-06 11:07:50 +02:00
Thomas Kriechbaumer
c0b12da401 change test value 2016-09-04 09:31:45 +02:00
Thomas Kriechbaumer
4c7f99d2be improve netlib.encoding 2016-09-04 09:31:44 +02:00
Thomas Kriechbaumer
69b770469e http2: assemble connect-request from object 2016-09-03 12:31:35 +02:00
Thomas Kriechbaumer
ea2a51d3be http2: fixes 2016-09-03 11:50:37 +02:00
Thomas Kriechbaumer
5dda9505b6 http2: improve framereader 2016-09-03 10:00:31 +02:00
Thomas Kriechbaumer
0badb3ec91 cleanup 2016-09-03 10:00:31 +02:00
Thomas Kriechbaumer
0d0c2c788d cleanup tests 2016-09-01 10:12:13 +02:00
Thomas Kriechbaumer
e5b0dae7e9 add websockets support to mitmproxy 2016-09-01 09:57:18 +02:00
Thomas Kriechbaumer
d12515f84b websockets: refactor implementation and add tests 2016-09-01 09:56:14 +02:00
arjun23496
a8deed1f4e Lint fix - minor 2016-08-31 16:39:04 +05:30
arjun23496
2476809ff8 Lint fix - replace tests 2016-08-31 16:27:22 +05:30
arjun23496
d2cfa5ca41 Added tests for replace in request,message,header 2016-08-31 10:23:33 +05:30
Manish Kumar
2cf4efab07 fixing lint error 2016-08-26 14:22:02 +05:30
Manish Kumar
16401d5be8 fixed : Divide by zero error came when string was empty, also a test 2016-08-26 12:52:51 +05:30
Thomas Kriechbaumer
51771c01c6 allow test classes to dynamically specify ssl context 2016-08-24 09:19:42 +02:00
Maximilian Hils
f27028f58e introduce Response.make for simple response creation 2016-08-23 00:17:06 -07:00
Thomas Kriechbaumer
eb83107e1c reenable options test 2016-08-16 11:05:57 +02:00
Thomas Kriechbaumer
cd35a6ff47 add regression test for #1476 2016-08-16 11:05:57 +02:00
Thomas Kriechbaumer
bfe22e739c Merge pull request #1481 from Kriechi/http2-simplify-zombies 
http2: simplify zombie detection
 2016-08-16 10:10:39 +02:00
Thomas Kriechbaumer
17c65e46cb Merge pull request #1480 from mhils/reverse-proxy-auth 
Add basic auth support for reverse proxy mode
 2016-08-16 10:07:59 +02:00
Shadab Zafar
239eaf0078 Remove timezone specific value 2016-08-15 15:05:32 +05:30
Shadab Zafar
bc175b58f2 Use test times less than 2038 
So that tests pass on 32bit systems too
 2016-08-15 12:00:23 +05:30
Shadab Zafar
ec0bae35c5 Assert cookie groups explicitly rather than just the length 2016-08-15 12:00:23 +05:30
Shadab Zafar
b9426fcec1 Add a test for base64 encoding 2016-08-15 12:00:23 +05:30
Shadab Zafar
988174246b Add some cookie formatting related tests 2016-08-15 12:00:23 +05:30
Shadab Zafar
456f80d862 Open JSON file in text mode 2016-08-15 12:00:23 +05:30
Shadab Zafar
03e6117042 Add a function to get cookie expiration time 2016-08-15 12:00:23 +05:30
Shadab Zafar
a479c51465 Add method to group pairs by cookies 2016-08-15 12:00:23 +05:30
Shadab Zafar
99b32094d8 Add simple har_dump test 2016-08-15 12:00:23 +05:30
Thomas Kriechbaumer
8f4baf42ea http2: cleanup 2016-08-14 13:06:33 +02:00
Maximilian Hils
a9401472cd add basic auth support for reverse proxy mode 2016-08-13 11:04:53 -07:00
Maximilian Hils
5a22496ee8 clean up code, improve DummyReply 2016-08-09 22:29:07 -07:00
Maximilian Hils
818840f553 finalize Reply semantics, fix tests 2016-08-09 20:26:24 -07:00
Maximilian Hils
dcfa7027ae fix tests 2016-08-04 15:39:48 -07:00
Maximilian Hils
951885a5dd simplify contentview logic 2016-08-04 15:23:44 -07:00
Maximilian Hils
7ea9726e5d Merge pull request #1453 from dufferzafar/limit-to-filter 
Rename "Limit" feature to "Filter View"
 2016-07-31 23:13:31 -07:00
Aldo Cortesi
ca0eab0f05 Merge pull request #1447 from mitmproxy/secure-by-default 
Verify upstream certificates by default
 2016-08-01 14:44:16 +12:00
Shadab Zafar
10ad56c853 Rename "Limit" feature to "Filter View" 2016-07-31 16:54:09 +05:30
Maximilian Hils
ca2e338876 Merge pull request #1449 from mhils/fix-1448 
Add escape_single_quotes=False arg to bytes_to_escaped_str
 2016-07-30 15:56:55 -07:00
Angelo Agatino Nicolosi
6792ec4058 Integrated encode/decoder for brotli 2016-07-30 14:43:53 +02:00
Maximilian Hils
4534363671 add escape_single_quotes=False arg to bytes_to_escaped_str 2016-07-29 19:54:44 -07:00
dufferzafar
1cffa5f46b Use replace while decoding 2016-07-28 07:48:10 -07:00
Maximilian Hils
f54b302a59 fix cert verification tests, remove leftovers 2016-07-27 22:44:06 -07:00
Maximilian Hils
8b325fd65a improve invalid certificate ux 2016-07-27 21:01:28 -07:00
Sachin Kelkar
17fdb841f0 verify upstream certificates by default (#1111) 
squashed and merged by @mhils
 2016-07-27 17:57:38 -07:00
Maximilian Hils
013288f7ca add quote/unquote tests 2016-07-24 19:38:44 -07:00
Maximilian Hils
e07f515a20 remove content caching in netlib.http.Message 2016-07-23 19:05:10 -07:00
Maximilian Hils
d6209fa03b add tests for single-element cache 2016-07-23 19:04:57 -07:00
Maximilian Hils
a87d089561 Merge pull request #1380 from dufferzafar/marked-filter 
Marked property and filter
 2016-07-23 17:55:12 -07:00
Maximilian Hils
fa45722ed8 Merge pull request #1402 from mhils/console-fixes 
minor fixes
 2016-07-23 12:49:57 -07:00
Maximilian Hils
61de6fa1d6 fix test_view_urlencoded 2016-07-23 11:55:27 -07:00
Thomas Kriechbaumer
514ea1b701 http2: cleanup tests 2016-07-23 11:08:32 +02:00
Shadab Zafar
08000c940c Test marked property while serialization roundtrip 2016-07-23 10:41:57 +05:30
Aldo Cortesi
afda175e1c Addon iface: .configure(options) -> .configure(options, updated) 2016-07-23 15:43:55 +12:00
Aldo Cortesi
b5416895f5 script: convert test recorder to an addon class 2016-07-23 11:38:45 +12:00
Aldo Cortesi
9294d19f90 scripts: .start can now return an Addon instance 2016-07-23 11:29:13 +12:00
Aldo Cortesi
51a8ba57f1 script: add a namespace adaptor to match Addons 2016-07-23 11:03:50 +12:00
Maximilian Hils
65c2f30218 fix mitmproxy 0.11 dumpfile - was 0.18 before 2016-07-22 10:36:26 -07:00
Maximilian Hils
798759d2b3 fix content view cache invalidation 2016-07-21 19:49:32 -07:00
Aldo Cortesi
33ffbc28ed console: share one ActionBar instance 
This fixes an issues where popping a view off the stack caused stale ActionBar
contents to be displayed.
 2016-07-21 15:41:43 +12:00
Aldo Cortesi
6750ab899a Merge branch 'flow-export' of https://github.com/dufferzafar/mitmproxy into dufferzafar-flow-export 2016-07-21 13:53:41 +12:00
Aldo Cortesi
02acfb1242 Fix netlib.utils.is_valid_host 
- Don't crash when passed an empty string. This translated into an actual core
crash, discovered while fuzzing with afl.
- Taking a slice of length one out of bytes returns an integer, so the check
for trailing period in this function never worked on Python3.
- Add unit tests.
 2016-07-21 10:38:37 +12:00
Shadab Zafar
c84d0bd10b Add a url exporter 2016-07-20 10:03:00 +05:30
Shadab Zafar
8f4d49e22a Remove raw_request exporter 
This functionality will be handled by headers+content
 2016-07-20 10:03:00 +05:30
Aldo Cortesi
8317772377 mitproxy.options -> mitmproxy.optmanager, mitmproxy.flow.options -> mitmproxy.options 
It might be slightly  more felicitous to move optmanager into netlib at some
point, especially if we can also use it in pathod.

This also consolidates our constants in mitmproxy.options, removing some duplicates.
 2016-07-20 11:28:53 +12:00
Aldo Cortesi
09fbebf42a Test suite: get_proxy_config -> get_options 2016-07-19 19:36:40 +12:00
Aldo Cortesi
9c9d28d068 Remove proxy.config.process_proxy_options 2016-07-19 16:25:09 +12:00
Aldo Cortesi
ef380917a8 ProxyConf: adapt mitmproxy console, ditch tcp|host filter attrs on flow.master 2016-07-19 16:25:09 +12:00
Aldo Cortesi
be081a24bc ProxyConf: auth_* into Options 2016-07-19 16:25:09 +12:00
Aldo Cortesi
5cfe783b6c ProxyConfig: http2, rawtcp, no_upstream_cert -> Options 2016-07-19 16:25:09 +12:00
Aldo Cortesi
26fa88a338 ProxyConfig: Refactor to move verification mode checks into configure 2016-07-19 16:25:09 +12:00
Aldo Cortesi
98bf544664 Regularize naming of upstream_trusted_ca* options 2016-07-19 16:25:09 +12:00
Aldo Cortesi
0a3839375d ProxyConfig: various SSL options to Options 2016-07-19 16:25:09 +12:00
Aldo Cortesi
78e8159007 config.parse_server_spec shouldn't pass byts to tcp.Address 2016-07-19 16:25:09 +12:00
Aldo Cortesi
f9622074cc ProxyConfig: mode, upstream_server and upstream_auth to Options 2016-07-19 16:25:09 +12:00
Aldo Cortesi
d2268ddb1e ProxyConfig: --body-size-limit to Options 2016-07-19 16:25:09 +12:00
Aldo Cortesi
856e1c2ba9 Move clientcers & cadir to Options 
Also adds a .configure mechanism to ProxyConfig, and hooks it up to the change
event.
 2016-07-19 16:25:09 +12:00
Aldo Cortesi
b1b1a1b9cf ProxyConfig: cadir to options 2016-07-19 16:25:09 +12:00
Aldo Cortesi
6908dc4d90 ProxyConfig refactor 
Step two of frog boiling: move listening address and port into options. This is
the hard bit, because it touches the test suite so deeply.
 2016-07-19 16:25:09 +12:00
Aldo Cortesi
9bc1514aef Unify proxy config and options 
This is step 1 in a gradual "boiling frog" strategy. Decorate ProxyConfig
with an Options object.
 2016-07-19 16:25:09 +12:00
Aldo Cortesi
e9865800a8 Console options cleanup 
Use setters, use togglers, move more things out of master
 2016-07-18 10:31:34 +12:00
Aldo Cortesi
f9e4249499 Add a toggller helper to Options, start using it in mitmproxy console 2016-07-18 10:03:10 +12:00
Thomas Kriechbaumer
7b57c2948e cleanup 2016-07-17 18:17:35 +02:00
Thomas Kriechbaumer
d9021f8505 http2: coverage++ 2016-07-17 18:07:02 +02:00
Aldo Cortesi
216e38d437 setheaders -> addon 2016-07-17 18:41:14 +12:00
Aldo Cortesi
a26cc75aab Merge pull request #1370 from cortesi/filestreamer 
Some work on stream-to-file
 2016-07-17 16:20:41 +12:00
Aldo Cortesi
9bf1d300d8 Some work on stream-to-file 
- stream -> filestreamer throughout
- move active_flows set into FileStreamer addon
 2016-07-17 16:07:29 +12:00
Aldo Cortesi
fb45d59c02 Merge pull request #1367 from Kriechi/fix-1366 
fix-1366
 2016-07-17 15:06:58 +12:00
Aldo Cortesi
08b3af98c2 Merge branch 'master' into dumper 2016-07-17 12:53:14 +12:00
Aldo Cortesi
b0b3b19ad6 Extract console dump functionality into an addon 
This removes all the code that deals with printing flows to screen from dump.py
into a self-contained addon.

- This fixes a bug - by moving dumping into an
addon, we now dump flows AFTER addon transformation, so we can see the changes
made.
- We get dumping "for free" in other places by simply adding the addon. It's
now easy to add dumping to console to mitmweb for debugging and development.
The same goes for external projects that derive from master.
- We also get major benefits in clarity for a frankly hairy part of our
project. Mitmdump is much clearer, and all the hairyness is now isolated for
further refactoring.
 2016-07-17 10:17:02 +12:00
Thomas Kriechbaumer
a32312cecc fix-1366 2016-07-16 18:31:37 +02:00
Maximilian Hils
afc3f6d854 Merge remote-tracking branch 'duffer/tcpflow-filter' 2016-07-16 00:19:01 -07:00
Maximilian Hils
b27d59095d Merge pull request #1306 from mitmproxy/message-body-encoding 
Improve Message Body Encoding
 2016-07-16 00:13:58 -07:00
Maximilian Hils
e6e39ce80f preserve content-type parameter order 2016-07-15 23:46:12 -07:00
Aldo Cortesi
1a1f7e6fd9 replacehooks -> addon 
Also fixes a bug in header replacements in netlib that resulted in a mutable
multidict.
 2016-07-16 18:37:30 +12:00
Maximilian Hils
3602fd7a36 Merge remote-tracking branch 'origin/master' into message-body-encoding 2016-07-15 23:17:57 -07:00
Maximilian Hils
a3c7c84d49 improve message content semantics 2016-07-15 22:50:33 -07:00
Shadab Zafar
fea4435dac Allow DummyFlow to match ~e, ~src and ~dst 2016-07-16 10:15:38 +05:30
Shadab Zafar
1d2ccb9170 Add DummyFlow and its Tests 2016-07-16 10:15:38 +05:30
Shadab Zafar
8c49f0e784 Add tests for TCPFlow.match 2016-07-16 10:15:38 +05:30
Shadab Zafar
6a2668d865 Add ~http and ~tcp filters 2016-07-16 10:15:38 +05:30
Shadab Zafar
262a420553 Use ~bq / ~bs for messages sent from client / server 2016-07-16 10:15:38 +05:30
Shadab Zafar
767457fda9 Add tests for HTTPFlow only filters 2016-07-16 10:15:38 +05:30
Shadab Zafar
317355a9af Add test for operators (& | !) 2016-07-16 10:15:38 +05:30
Shadab Zafar
4e29993c1f Add test for addresses (~src, ~dst) 2016-07-16 10:15:38 +05:30
Shadab Zafar
124b87b851 Add test for body (~b) 2016-07-16 10:15:38 +05:30
Shadab Zafar
88671c3aac Add test for error (~e) 2016-07-16 10:15:38 +05:30
Aldo Cortesi
5f23d4f5ca add_event -> add_log throughout project 
"Event" is terribly over-loaded in the project, and "log" is straight-forward
and self-explanatory.
 2016-07-16 10:19:06 +12:00
Aldo Cortesi
c4fb489a96 Zap unused test scripts 2016-07-15 22:37:52 +12:00
Aldo Cortesi
c7d0850d8f Script cleanups 
- Preserve script order on config change
- Prohibit script duplicates (i.e. identical script + args)
- Various cleanups and tweaks
 2016-07-15 16:35:24 +12:00
Aldo Cortesi
917d51bd22 Fix HAR extractor 
Thanks to @mhils
 2016-07-15 15:01:35 +12:00
Aldo Cortesi
92a5076bd0 Adjust concurrent tests for start method 2016-07-15 15:00:37 +12:00
Aldo Cortesi
4ba7ce50c6 Add .start for addons and scripts 
Also improve error messages, fix various unit tests
 2016-07-15 14:52:34 +12:00
Aldo Cortesi
fcc1416ffd Fix windows parse_args test 2016-07-15 10:33:51 +12:00
Aldo Cortesi
698af945a4 Adjust for new options scheme 2016-07-14 19:58:12 +12:00
Aldo Cortesi
a4127fb6d5 Add a tick event for addons and scripts, and use it for race-free reload 2016-07-14 19:54:15 +12:00
Aldo Cortesi
deffed2196 Script cleanup: editing in console, Python3 compatibility fixes 2016-07-14 19:54:15 +12:00
Aldo Cortesi
b94f5fd361 Convert examples and example tests for new-style scripts 
Remove the test that just loads all the example scripts for now - it's a very
low-value test, and we need to think of something better.
 2016-07-14 19:54:15 +12:00
Aldo Cortesi
a6821aad8e Zap old scripts infrastructure, fix concurrency tests 2016-07-14 19:54:15 +12:00
Aldo Cortesi
a3a22fba33 First-order integration of scripts addon 2016-07-14 19:54:15 +12:00
Shadab Zafar
1266255842 Re-add deletion of stickycookies (#1355) 
This was originally added in #1324 (fixing issue #1096)
but got lost during "the big refactor" (#1352)
 2016-07-14 00:37:27 -07:00
Maximilian Hils
f84a23f11d minor fixes 2016-07-13 22:01:16 -07:00
Maximilian Hils
ec6fbe9eb6 make options explicit 2016-07-13 21:45:25 -07:00
Aldo Cortesi
b2c4f301cb Stream to file -> addon 
This commit also clarifies a confusion about the "outfile" attribute and its
use in testing in the mitmdump master.
 2016-07-14 13:52:09 +12:00
Aldo Cortesi
703c05066e Fix indeterminacy in sticky cookie tests 
How has this not bitten us before?
 2016-07-14 12:59:00 +12:00
Aldo Cortesi
00509d86a8 StickyCookies to addon 2016-07-14 12:13:14 +12:00
Aldo Cortesi
143bf0dfa9 AntiCache to addon 2016-07-14 11:24:06 +12:00
Aldo Cortesi
97b2e6d958 Add a .setter helper to Options 
This returns a function that sets a named attribute, and is handy for
event-driven code like mitmproxy console.
 2016-07-14 11:11:41 +12:00
Aldo Cortesi
a8a083a10d Move StickyAuth into addon 
This is the first addon that handles an OptionsError, so this commit also demos
how this works in console. Handling of command-line erorrs is on its way.
 2016-07-14 10:08:41 +12:00
Aldo Cortesi
255e1eb00b Add the addons mechanism 
Demonstrate how it works and interacts with Options by adding our first new
builtin: anticomp.
 2016-07-13 23:45:49 +12:00
Aldo Cortesi
7f8fd3cdff Basic outline of addons 
Add addons.py, integrate with our event mechanism, and change the Master API so
options is the first init argument.
 2016-07-13 23:26:04 +12:00
Aldo Cortesi
a20f8e9620 More powerful Options scheme 
This prepares us for the addon configuration mechanism and gives us a more
flexible way to handle options changes. This changeset should spell the end of
the current anti-pattern in our codebase where we duplicate data out of options
onto the master when mutability is needed. From now on, Options can be the one
source of thruth.

- Change notifications
- Rollback on error
 2016-07-13 18:45:50 +12:00
Thomas Kriechbaumer
3579c6dd43 http2: fix race-condition in tests 2016-07-12 19:55:28 +02:00
Thomas Kriechbaumer
7c09255298 http2: more coverage 2016-07-10 20:07:43 +02:00
Thomas Kriechbaumer
b611997619 http2: simplify test class 2016-07-10 18:53:00 +02:00
Shadab Zafar
7eade1ef7c Remove redundant tests 2016-07-10 01:08:02 +05:30
Shadab Zafar
39f5108400 Test cookies.is_expired separately 2016-07-10 01:06:50 +05:30
Shadab Zafar
6a746deff5 Add tests for deletion of stickycookies 2016-07-09 22:20:25 +05:30
Shadab Zafar
83a1cc5a9a Make escape_control_characters handle strings on Py2 2016-07-09 13:28:20 +05:30
Shadab Zafar
c7cbbf5f0d Set verbosity & flow_detail to 0 2016-07-09 12:32:41 +05:30
Shadab Zafar
147f61fa62 py3++ 2016-07-09 11:14:12 +05:30
Maximilian Hils
15c10d49f6 Merge pull request #1327 from mitmproxy/remove-script-context 
Remove script context
 2016-07-08 20:11:12 -07:00
Maximilian Hils
5d2b7c52f9 move script context to mitmproxy.ctx 2016-07-08 19:57:57 -07:00
Maximilian Hils
f7639e077a py3++ 2016-07-08 15:58:12 -07:00
jpkrause
05ab9c1b37 Updated coding style and modified flow format compat test to use v0.11 and v0.10 dump files 2016-07-08 15:03:36 -07:00
Maximilian Hils
7c67faa8da remove script contexts 2016-07-07 23:50:55 -07:00
Maximilian Hils
c048ae1d5b remove context from all scripts 2016-07-07 20:14:52 -07:00
Maximilian Hils
00dce24015 tests++ 2016-07-07 18:02:59 -07:00
Maximilian Hils
76473d44e0 py3++ 2016-07-07 17:50:26 -07:00
Maximilian Hils
f62e976e1e py3++ 2016-07-07 17:29:22 -07:00
Thomas Kriechbaumer
d962bd4e83 py3++ 2016-07-07 20:25:48 +02:00
Shadab Zafar
c870d26bc7 py3++: test_protocol_http2 2016-07-07 16:39:25 +02:00
Maximilian Hils
f259b9478f disable harparser on py3 2016-07-07 01:21:15 -07:00
Thomas Kriechbaumer
64ac2c73fe Merge pull request #1318 from mhils/tnetstring3-unicode-type 
[tests actually pass] Add unicode to tnetstring3
 2016-07-07 09:48:22 +02:00
Linmiao Xu
275cf7a6c2 Detect JSON API media type as a json content view 
IANA assignment here:
https://www.iana.org/assignments/media-types/application/vnd.api+json

More about JSON API:
https://github.com/json-api/json-api
 2016-07-06 21:10:35 -07:00
Maximilian Hils
64a867973d sni is now str, not bytes 2016-07-06 21:03:17 -07:00
Maximilian Hils
8287ce7e6d fix tests 2016-07-06 19:55:48 -07:00
Maximilian Hils
d406bee988 tnetstring3: adapt to unicode support 2016-07-06 19:55:48 -07:00
Maximilian Hils
684b4b5130 tnetstring: keys are str on py3. migrate flow.io_compat 2016-07-06 19:55:48 -07:00
Maximilian Hils
9c873d63f4 py3++, multidict fixes 
This commit improves Python 3 compatibility and fixes two multidict
issues:

 1. Headers.items(multi=True) now decodes fields
 2. MultiDict.clear(item) has been removed, as Python's
    MutableMapping already defines .clear() with different
    semantics. This is confusing for everyone who expects a
    dict-like object. `.pop("attr", None)` is not fantastic,
    but it's the Python way to do it.
 2016-07-06 19:53:29 -07:00
Maximilian Hils
444f0a4c39 py3++ 2016-07-06 17:31:08 -07:00
Maximilian Hils
e84f5fcc72 Merge pull request #1315 from mitmproxy/better-unicode-handling 
remove clean_bin, clarify unicode handling
 2016-07-05 16:02:37 -07:00
Maximilian Hils
00a35aa895 Merge pull request #1297 from dufferzafar/py3-flow-export 
Python 3 - test_flow_export
 2016-07-05 15:17:18 -07:00
Maximilian Hils
d51cf543bb remove clean_bin, clarify unicode handling 2016-07-05 15:11:32 -07:00
Thomas Kriechbaumer
5fecb8c843 http2: test fatal connection termination 2016-07-05 22:15:57 +02:00
Thomas Kriechbaumer
8d2042ea8a http2: test throttling at MAX_CONCURRENT_STREAMS 2016-07-05 22:15:57 +02:00
Maximilian Hils
a6b3551934 raise ValueError if content-encoding is invalid 2016-07-04 13:58:09 -07:00
Shadab Zafar
45aa2174e2 http2: Remove TestReadRequestConnect test 2016-07-03 21:14:01 +05:30
Shadab Zafar
f623b3d99b py3++: test_flow_export 2016-07-02 21:15:20 +05:30
Shadab Zafar
393c1d74af fix lint issues 2016-07-02 18:56:22 +05:30