Grasscutters/mitmproxy
3
0
Fork 0
mirror of https://github.com/Grasscutters/mitmproxy.git synced 2024-11-30 03:14:22 +00:00
Code Issues Projects Releases Wiki Activity
7,193 Commits 1 Branch 67 Tags 50 MiB
21324086c3
Commit Graph

1707 Commits

Author SHA1 Message Date
Aldo Cortesi
1410cbb4b6 Remove test handlers by using taddons.RecordingMaster 
This also means expanding and tweaking the recording master API, which we
reflect through the current test suite
 2017-03-16 18:05:57 +13:00
Aldo Cortesi
b6abe96202 Move dump handlers to addon 2017-03-16 18:05:57 +13:00
Aldo Cortesi
1e6c51c698 Merge pull request #2152 from cortesi/readfile 
Extract flow reading into addons
 2017-03-16 15:51:21 +13:00
Aldo Cortesi
228a22b3c0 Add a light-weight custom event system, use it for keepserving 
This patch implements the lightweight event system I propose in #2144, adds a
custom event "processing_complete" that is triggered after file read, client
replay and server replay, and introduces a KeepServing addon to handle this for
mitmdump.
 2017-03-16 11:07:00 +13:00
Aldo Cortesi
169068c7ec Clean up addonmanager interface 
Clarify the plethora of invocation methods we've sprouted, correct some usages
in the codebase.
 2017-03-16 08:50:29 +13:00
Maximilian Hils
65c4a3cf1d add Request.make 2017-03-15 14:06:00 +01:00
Aldo Cortesi
ef582333ff Extract flow reading into addons 
This patch moves the final pieces of master functionality into addons.

- Add a ReadFile addon to read from file
- Add a separate ReadStdin addon to read from stdin, only used by mitmdump
- Remove all methods that know about io and serialization from master.Master
 2017-03-15 13:44:48 +13:00
Maximilian Hils
30797755fb stateobject: automatically change id when copying 2017-03-14 17:08:40 +01:00
Maximilian Hils
375680a3be add connection ids 2017-03-14 17:08:40 +01:00
Maximilian Hils
2d016775fe Merge pull request #2149 from mhils/multidictview-copy 
Add MultiDictView.copy()
 2017-03-14 14:38:38 +01:00
Maximilian Hils
0de8f46c6e Merge pull request #2145 from mhils/ne 
py3: __ne__ delegates to __eq__ by default
 2017-03-14 14:38:00 +01:00
Maximilian Hils
961747c4bc Merge pull request #2120 from ujjwal96/replace-from-file 
Replace from file through replacement editor #1458
 2017-03-14 02:23:31 +01:00
Maximilian Hils
de1de3f626 add MultiDictView.copy 2017-03-14 02:18:02 +01:00
Maximilian Hils
875ce8c9c1 py3: __ne__ delegates to __eq__ by default 2017-03-14 00:40:15 +01:00
Aldo Cortesi
b745428b5c Enable custom options in config files 
We also now ignore unknown options in config files by default, and print a
warning if verbosity is incremented.
 2017-03-14 09:40:33 +13:00
Aldo Cortesi
2832e790fd Move serialization methods out of Options 
Attributes on options share a namespace with options themselves. It's getting
too crowded on our Options object, so let's shift some obvious stuff into the
module.
 2017-03-14 08:34:25 +13:00
Aldo Cortesi
c24f7d8e12 Optmanager: handle unknown options in value sets 2017-03-14 08:32:19 +13:00
Aldo Cortesi
75f83ba380 Show custom addon options in --options output 2017-03-14 08:32:19 +13:00
Aldo Cortesi
0c6663d0d5 Enable custom options for addons 
- Add an options parameter to the start() event. This is to be used by addons
on startup to add custom options.
- Add a running() event that is called once the proxy is up and running.
- With the new paradigm we can't log during master __init__, so add a tiny
termstatus addon to print proxy status to terminal once we're running.
 2017-03-14 08:32:19 +13:00
Maximilian Hils
05e11547f5 Merge remote-tracking branch 'origin/master' into pr-2120 
Conflicts:
	test/mitmproxy/addons/test_replace.py
 2017-03-12 23:33:49 +01:00
Maximilian Hils
0f4b523868 replacements: error right away when file does not exist 2017-03-12 23:28:49 +01:00
Thomas Kriechbaumer
1b045d24bc nuke tutils.tmpdir, use pytest tmpdir 2017-03-12 22:55:22 +01:00
Maximilian Hils
4684617d2c minor fixes 2017-03-12 20:24:50 +01:00
Maximilian Hils
39154e628e Merge pull request #2118 from lymanZerga11/patch-3 
Update proxyauth.py to add proxyauth metadata
 2017-03-11 03:04:38 +01:00
Thomas Kriechbaumer
e9746c5182 Merge pull request #2114 from mitmproxy/fix-websocket-serialization 
make websocket flows serializable
 2017-03-10 21:15:46 +01:00
Thomas Kriechbaumer
45bf1ff64d disable h2c prior knowledge connections 2017-03-10 11:37:25 +01:00
Thomas Kriechbaumer
b1dd86d7ae make websocket flows serializable 
fixes #2113
 2017-03-09 19:08:59 +01:00
Ujjwal Verma
25703f8c53 Fixed Replacement test 2017-03-09 23:33:11 +05:30
lymanZerga11
8e67c6f512 add tests 2017-03-09 09:01:37 +08:00
lymanZerga11
6d53d08b6c Add tests 2017-03-09 09:01:04 +08:00
Maximilian Hils
63179d9751 core -> core_option_validation 
longer, but much clearer for devs who are unfamiliar with the codebase.
 2017-03-08 15:17:07 +01:00
Maximilian Hils
f276c7a80d change add_option parameter order 
name, type, value matches natural language ($x is a bool with a default value of $y)
and also matches the python type annotation order of name: type = value
 2017-03-08 00:52:37 +01:00
Maximilian Hils
8707928b16 unify server spec parsing 2017-03-08 00:18:34 +01:00
Aldo Cortesi
98ec3b77fe Carefully re-add a small number of short flags 
The mechanism for booleans attaches the short flag to whatever the opposite of
the default is.
 2017-03-07 21:01:21 +13:00
Aldo Cortesi
aed780bf48 Change the way proxy authetication is specified 
We now have one option "proxyauth". If this is "any", we accept any
credentials, if it starts with an @ it's treated as a path to an htpasswd file,
if it is of the form username:password it's a single-user credential.
 2017-03-07 20:18:30 +13:00
Aldo Cortesi
320d8848ab Options tweaks 
- Regularise presentation and wording
- Help is mandatory for all options
- Auto-generate wording to say that sequence options can be passed multiple
times on the command-line
 2017-03-07 15:23:46 +13:00
Aldo Cortesi
10db254791 Remove less commonly used command-line options 
We now have --set, so only options that really deserve it get a dedicated flag.
I'm inclined to strip this back even more. Feel free to argue that YOUR
favourite option deserves special treatment here.
 2017-03-07 14:46:28 +13:00
Aldo Cortesi
ac3b0d69cc Add the --set option to set options directly 
The --set option is a universal flag for setting options. Some examples:

Turn on a boolean:

mitmdump --set onboarding=false

Add a value to a sequence:

mitumdupm --set setheaders=/foo/bar/voing

Zero a sequence:

mitumdupm --set setheaders
 2017-03-07 14:27:50 +13:00
Aldo Cortesi
79f5883c2f Option spacing and coverage 2017-03-07 13:47:39 +13:00
Aldo Cortesi
99a6b0dbc1 Add --options that dumps annotated option defaults 2017-03-07 13:16:28 +13:00
Aldo Cortesi
b51df9a0b1 Cleanups and test coverage 2017-03-07 10:28:15 +13:00
Aldo Cortesi
2312cf6fb0 Automate slurping up command-line options 
Now that options are completely regular, we can automate reading them out of
arguments, rather than listing them out by hand like savages.
 2017-03-07 08:49:19 +13:00
Aldo Cortesi
edfd62e42a Replacements and setheaders are always strings 
Instead of having two representations we have one canonical specification.
Fixing the editor in console is left ot a further patch.
 2017-03-06 20:58:51 +13:00
Aldo Cortesi
82163a1e68 Unify mode specification 
We now have:

    --mode regular (the default)
    --mode transparent
    --mode socks5
    --mode reverse:SPEC
    --mode upstream:SPEC

Where SPEC is a host specification.
 2017-03-06 20:30:49 +13:00
Aldo Cortesi
9b1f40da37 Options unification: streamfile 
We now have one option to control this. If the path is prefixed with a "+" we
append, otherwise we overwrite.
 2017-03-06 13:48:37 +13:00
Aldo Cortesi
71a830c836 Options unification: string choices 2017-03-06 13:20:17 +13:00
Aldo Cortesi
18a6b66ba4 Options unification: simple strings 
Move all simple string options to the new scheme. Also regularise some names.
 2017-03-06 11:39:19 +13:00
Aldo Cortesi
201c65960e Options unification: sizes 
Start dealing with corner cases:

- Sizes are always stored in options as strings
- Add a new core addon that's responsible for verifying settings that don't
belong to an addon
- Add a _processed scratch space on the Options object for processed core
values to be stored in. This is pretty dirty, but less dirty than re-parsing
values every time. We'll come up with something better down the track.
 2017-03-06 09:07:16 +13:00
Aldo Cortesi
f15a628561 Start unifying options and the command-line: booleans 
This commit:

- Adds a help field to options
- Adds a function to generate parser definitions from options
- Uses this to migrate all boolean flags over to the new system
- Makes all booleans consistently follow the --foo/--not-foo convention

There are a number of things left to be done here:

- Argparse doesn't give us a nice way to format --foo --not-foo help. Click
does, and moving to click is a goal down the track.
- For now, we remove all short aliases. I want to re-evaluate these
systematically once we have the new structure in place.
 2017-03-05 20:16:33 +13:00
Aldo Cortesi
67381ae550 Revamp options 
- Options are now explicitly initialized with an add_option method
- We have one canonical Options class - ditch dump.Options
 2017-03-05 15:10:03 +13:00
Nikhil Soni
0081d9b828 Merge branch 'master' into on-issues 2017-03-03 12:58:44 +05:30
Nikhil Soni
3da8532bed Adds test for --keep-host-header 2017-03-03 12:34:36 +05:30
Nikhil Soni
317d183ba4 Changes dns_spoofing example to use --keep-host-header 2017-03-03 12:34:36 +05:30
Thomas Kriechbaumer
a505221e1a http2: fix h2 header normalization test (#2080) 2017-03-01 12:46:42 +01:00
David Dworken
99b584ad7d added XSS scanner example (#1907) 2017-02-27 18:22:39 +01:00
Thomas Kriechbaumer
e0bc1109c0 do not return IP addresses in SSLCert.altnames 2017-02-26 20:54:06 +01:00
Thomas Kriechbaumer
6d228dfb9f certs: coverage++ 2017-02-26 20:54:04 +01:00
Thomas Kriechbaumer
9b6986ea87 connections: coverage++ (#2064) 2017-02-26 20:50:52 +01:00
Thomas Kriechbaumer
2aecffd39a optmanager: coverage++ (#2062) 2017-02-25 12:12:43 +01:00
Thomas Kriechbaumer
ccd8e1e617 http: coverage++ (#2060) 2017-02-24 22:39:16 +01:00
Thomas Kriechbaumer
3c66eb5b4b move tests around (#2059) 2017-02-24 22:00:41 +01:00
Thomas Kriechbaumer
e89c785719 move tests around (#2058) 2017-02-24 21:39:13 +01:00
Thomas Kriechbaumer
d17309eda8 flowfilter: coverage++ 2017-02-24 18:47:56 +01:00
Thomas Kriechbaumer
3e9125a3c1 nuke tcp.Address and add proper IPv6 support 2017-02-24 13:39:02 +01:00
Maximilian Hils
ba76dbc672 Merge pull request #2048 from ujjwal96/ipv6-addresses 
Absolute IPv6 addresses supported
 2017-02-22 13:17:53 +01:00
Matthew Shao
5fc4fc28b6 Add test for @concurrent decorator in class. 2017-02-21 16:56:48 +08:00
Ujjwal Verma
11134b669e Absolute IPv6 addresses supported 2017-02-20 21:02:43 +05:30
Thomas Kriechbaumer
0de277c18b improve test 2017-02-20 11:48:57 +01:00
Thomas Kriechbaumer
050245e842 Merge pull request #2017 from lymanZerga11/patch-1 
Catch ValueErrors from url.parse()
 2017-02-20 11:48:40 +01:00
Thomas Kriechbaumer
ac6915c002 improve tests 2017-02-20 11:11:10 +01:00
Thomas Kriechbaumer
51e08a5b85 Merge pull request #2041 from krsoninikhil/debug 
Adds PyInstaller build indicator to --version argument
 2017-02-20 10:29:15 +01:00
Nikhil Soni
ed084f5413 Adds test for pyinstaller indicator 2017-02-19 19:15:36 +05:30
Thomas Kriechbaumer
36352c9539 protobuf: coverage++ 2017-02-19 14:29:09 +01:00
Thomas Kriechbaumer
83c2de8849 http2: disable priority forwarding 2017-02-18 14:30:08 +01:00
Maximilian Hils
4158a1ae55 Merge pull request #2040 from mhils/request-host-header 
Add "Request.host_header"
 2017-02-18 12:08:54 +01:00
Thomas Kriechbaumer
6ef4f094b3 Merge pull request #2038 from Kriechi/improve-http2-cov 
test forbidden HTTP/2 headers in reponses
 2017-02-18 11:50:57 +01:00
Thomas Kriechbaumer
47e6f977de test forbidden HTTP/2 headers in reponses 2017-02-18 11:26:54 +01:00
Maximilian Hils
49c8e19f80 add request.host_header, fix #2036 2017-02-17 23:31:53 +01:00
Maximilian Hils
6a1e03ac6f tests: make full cov plugin cwd-independent 2017-02-17 23:29:10 +01:00
Maximilian Hils
33acb48c71 Merge pull request #2032 from Kriechi/individual-coverage 
add individual-coverage check
 2017-02-17 22:56:13 +01:00
Thomas Kriechbaumer
4cec88fc7f remove unneeded test module 2017-02-17 20:41:10 +01:00
mame82
ffdbccd571 Use existing urlencoding conventions when re-encoding edited form #1946 (#2022) 2017-02-16 15:03:27 +01:00
Thomas Kriechbaumer
6b22ca7a32 add individual-coverage check 2017-02-16 11:43:45 +01:00
Thomas Kriechbaumer
337b1c9399 fix wrong filename 2017-02-15 18:45:49 +01:00
Maximilian Hils
d1186eea18 Merge pull request #2029 from mhils/proxyauth 
Improve ProxyAuth
 2017-02-15 16:52:20 +01:00
Thomas Kriechbaumer
94a7e99fda Merge pull request #2028 from Kriechi/nuke-pillow 
nuke Pillow
 2017-02-15 15:29:57 +01:00
Maximilian Hils
4bac850bb1 fix #1722, fix #1734, refs #2019 2017-02-15 14:20:46 +01:00
Maximilian Hils
0d9c7ce50c image view: add fallback, catch all images but svgs 2017-02-15 14:04:24 +01:00
Thomas Kriechbaumer
8a6f8bd461 nuke Pillow 2017-02-15 00:54:14 +01:00
Thomas Kriechbaumer
24a51df9cb extract full-coverage pytest plugin 2017-02-15 00:29:43 +01:00
Thomas Kriechbaumer
4d973e8295 fix imports 2017-02-14 23:47:33 +01:00
Thomas Kriechbaumer
a12c3d3f8e restructure and move test files 
add empty test files to satisfy linter
 2017-02-14 23:31:37 +01:00
Thomas Kriechbaumer
04748e6f3f add filename-matching linter 2017-02-14 22:48:24 +01:00
lymanZerga11
d30ef7ee3e Update test_flowlist.py 2017-02-14 21:17:18 +08:00
lymanZerga11
a52d8c1dab Update test_flowlist.py 2017-02-14 21:01:01 +08:00
lymanZerga11
26a17a3d82 Update test_flowlist.py 2017-02-13 15:05:29 +08:00
lymanZerga11
a912d67c06 Update test_flowlist.py 2017-02-13 10:47:50 +08:00
lymanZerga11
577fb818b9 Update test_flowlist.py 2017-02-13 10:39:48 +08:00
lymanZerga11
f5b30b8872 Update test_flowlist.py 2017-02-12 23:36:26 +08:00
lymanZerga11
df8a5aa9be Update test_flowlist.py 2017-02-12 23:23:23 +08:00
lymanZerga11
c622e4a649 Create test_flowlist.py 2017-02-12 23:10:49 +08:00
Maximilian Hils
3067a971f9 Merge pull request #2000 from s4chin/add-jpeg-parser 
Add jpeg parser
 2017-02-11 23:00:14 +01:00
Sachin Kelkar
245e24dcf3 Add sources of images, remove copyrighted image 2017-02-12 01:19:36 +05:30
Maximilian Hils
71742654e3 Merge pull request #2005 from ujjwal96/script-reload 
Closes #1828 script reloads on py file changes
 2017-02-11 01:45:14 +01:00
Thomas Kriechbaumer
c8c79cc291 coverage++ 2017-02-10 22:55:15 +01:00
Sachin Kelkar
8eb1d34644 Add app1 marker and tests 2017-02-10 22:40:19 +05:30
Ujjwal Verma
809207195d closes #1828 script reloads on py file changes 2017-02-10 22:21:12 +05:30
Sachin Kelkar
c4e141a000 Add jpeg app0 sof0 com parsing 2017-02-10 11:16:23 +05:30
Ujjwal Verma
d6465b907f Closes #1580 Gives warning when explicit proxy configured in transparent mode (#1996) 2017-02-09 20:56:06 +01:00
Maximilian Hils
380ff50e57 Merge pull request #2003 from Kriechi/coverage++ 
test refactoring and coverage++
 2017-02-09 17:45:50 +01:00
Maximilian Hils
da8444b11f flow_count() -> len(flows) 2017-02-09 16:44:27 +01:00
Maximilian Hils
1084588103 Merge pull request #1998 from mhils/fix-change-upstream 
Fix change_upstream_proxy_server
 2017-02-09 16:39:20 +01:00
Thomas Kriechbaumer
d10560d54c improve coverage report 2017-02-09 16:08:36 +01:00
Thomas Kriechbaumer
2ff5d72236 minor changes 2017-02-09 16:08:36 +01:00
Thomas Kriechbaumer
0299bb5b2e eventsequence: coverage++ 2017-02-09 16:08:36 +01:00
Thomas Kriechbaumer
5a3976c43e coverage whitelist -> blacklist 2017-02-09 16:08:36 +01:00
Thomas Kriechbaumer
f6cea09d5a stateobject: coverage++ 2017-02-09 16:08:36 +01:00
Thomas Kriechbaumer
1847cf175c websockets, tcp, version: coverage++ 2017-02-09 16:08:35 +01:00
Thomas Kriechbaumer
ae94ca6fa9 remove deprecated flow_count function 2017-02-09 16:08:35 +01:00
Maximilian Hils
9b97b63891 fix change_upstream_proxy_server 2017-02-09 15:24:31 +01:00
Maximilian Hils
7a205e80aa improve stateobject tests 2017-02-09 14:21:25 +01:00
Thomas Kriechbaumer
7a9d40817c pytest.raises: shim new API 2017-02-08 20:04:07 +01:00
Maximilian Hils
28c0596742 Merge pull request #1997 from s4chin/add-gif-parser 
Add gif parser and tests
 2017-02-08 12:09:48 +01:00
Sachin Kelkar
66ad95c330 Fix gif parsing error on some gifs and add test 2017-02-08 00:03:07 +05:30
Sachin Kelkar
4ef5de2cce Add gif parser and tests 2017-02-06 17:57:59 +05:30
Ujjwal Verma
c622622c59 Encoding fixes and tests 2017-02-05 18:59:01 +05:30
Thomas Kriechbaumer
288f9a3857 fix flaky test 2017-02-05 10:42:47 +01:00
Thomas Kriechbaumer
966ffaa3d6 coverage++ 2017-02-04 13:48:55 +01:00
Maximilian Hils
53f298ac41 Merge pull request #1967 from s4chin/remove-pillow 
Add png parser
 2017-02-03 18:58:34 +01:00
Maximilian Hils
bbdb7300fd Merge pull request #1983 from Kriechi/coverage++ 
coverage++
 2017-02-03 18:56:56 +01:00
Sachin Kelkar
15548ff433 Feedback 2017-02-03 18:40:50 +05:30
Thomas Kriechbaumer
bcaaa2f40b coverage++ 2017-02-03 11:55:33 +01:00
Sachin Kelkar
0674485e76 Fix and cleanup 2017-02-03 15:02:35 +05:30
Sachin Kelkar
88dbfd5257 Add iTXt and zTXt tests 2017-02-03 14:12:54 +05:30
Sachin Kelkar
abef020e07 Fix as per feedback and add more tests 2017-02-03 14:12:54 +05:30
Sachin Kelkar
6202958048 Add png parser 2017-02-03 14:12:54 +05:30
Ujjwal Verma
e10c36fe11 Added test 2017-02-03 02:58:02 +05:30
Thomas Kriechbaumer
79aa994275 fix merge 2017-02-02 17:26:31 +01:00
Thomas Kriechbaumer
4f0b2bc4de adapt coverage checks 2017-02-02 14:20:56 +01:00
Thomas Kriechbaumer
ae008ed80b replace tutils.raises with pytest.raises + shim 2017-02-02 12:59:01 +01:00
Thomas Kriechbaumer
ec92d7f67e cleanup test utils 2017-02-02 12:59:01 +01:00
Thomas Kriechbaumer
6e329595ca add test coverage protection 2017-02-02 11:39:33 +01:00
Thomas Kriechbaumer
380d8ec370 increase test coverage 2017-02-02 10:15:01 +01:00
Thomas Kriechbaumer
3ae060f0d3 fix console test 2017-02-02 10:13:12 +01:00
Maximilian Hils
7b2a986cea tests++ (#1976) 2017-02-01 17:52:01 +01:00
lymanZerga11
4b10212caf Add except clause to catch script parsing errors (#1929) 2017-02-01 15:27:10 +01:00
Maximilian Hils
212d9f1b98 Merge pull request #1948 from amm0nite/fix_dns_spoofing_example 
Fix for dns_spoofing.py example
 2017-02-01 15:25:51 +01:00
Aldo Cortesi
914659e888 Regularise setheaders options 
As per replacements:

- Make the option type a string/tuple union
- Localise parsing strictly within the addon
- Adapt the console editor (we'll find a more elegant solution later)
 2017-02-01 12:22:05 +13:00
Ammonite
9e3f06b7f2 Better test 2017-01-31 23:23:13 +01:00
Aldo Cortesi
4cc75a9560 Revamp replacement hooks 
- Replacement specifiers can be either strings or tuples. This lets us cope
gracefully with command-line parsing (and posible quick interactive
specification) without having to special-case replacement hooks, or have
knowledge of hook specification leak outside the addon. We can also now use the
same command-line spec format in config files.
- Split replacement and replacement from file into separate addons and options.
Files are now read on each replacement, so you can edit replacement files in
place without restart.
- Modernise the test suite to use addon test helpers.

TODO: editing and displaying replace-from-file in console app
 2017-02-01 11:10:28 +13:00
Aldo Cortesi
02f51d043d Merge pull request #1965 from cortesi/optname 
Consolidate option names
 2017-02-01 08:39:35 +13:00
Thomas Kriechbaumer
ff379b7665 fully test addonmanager (#1970) 2017-01-31 10:57:22 +01:00
Ammonite
e2e15df358 Add a test for the host replacement 2017-01-30 23:44:13 +01:00
Aldo Cortesi
35aff3b783 Consolidate option names 
- Onboarding app options to onboarding*
- Console-specific options to console_*
- Web-specific options to web_*
 2017-01-30 11:15:12 +13:00
Aldo Cortesi
356c4987a2 addons: unit tests to 100% 
This patch pushes the coverage of ./tests/mitproxy/addons to 100% of the addons
module.
 2017-01-29 11:44:49 +13:00
Thomas Kriechbaumer
d24f76c98e fix newlines on windows 2017-01-25 18:51:09 +01:00
Thomas Kriechbaumer
5549757268 fix alpn requirement in web tests 2017-01-25 18:18:18 +01:00
Thomas Kriechbaumer
3028e06fd2 simplify termlog outfile handling 2017-01-25 17:50:07 +01:00
nish21
ab45e4d183 write errors to stderr (#1952) 2017-01-25 17:49:56 +01:00
Maximilian Hils
2eaac31344 Merge pull request #1945 from Kriechi/ca-expired 
fix #939
 2017-01-23 21:22:38 +01:00
Thomas Kriechbaumer
5792e2c483 test disable_h2c_upgrade addon 2017-01-23 18:38:27 +01:00
Thomas Kriechbaumer
d674de298c test websocket addon 2017-01-23 15:45:53 +01:00
Thomas Kriechbaumer
006eb39cc5 fix #939 2017-01-23 15:14:56 +01:00
Thomas Kriechbaumer
127c69c3ac nuke mock dependency 
This is already included in Python 3.3+ as unittest.mock
 2017-01-22 14:11:28 +01:00
Thomas Kriechbaumer
72b753c60f provide git information with --version 
fixes #1848
 2017-01-19 20:27:29 +01:00
Maximilian Hils
88f3459c7d stronger tests, lint 2017-01-07 23:08:50 +01:00
Maximilian Hils
b1ec7e78cd Revert "add strutils.replace_surrogates" 
This reverts commit 15ae88db08dbf2725c8c9f945f2006c4b2a10d46.
 2017-01-07 23:08:50 +01:00
Maximilian Hils
c21ee90deb add strutils.replace_surrogates 2017-01-07 23:08:50 +01:00
Maximilian Hils
042261266f minor encoding fixes 
- native() -> always_str()
  The old function name does not make sense on Python 3 only.
- Inline utility functions in message.py.
 2017-01-07 23:08:50 +01:00
lymanZerga11
3ea76a7f3e Controls the length of URL displayed using the -d option (#1903) 2017-01-05 14:37:20 +01:00
chhsiao90
a5f1215eb2 Allow underscore in hostname 2016-12-30 21:03:22 +08:00
Maximilian Hils
eab360a02b fix IO type checking 2016-12-28 14:21:19 +01:00
Maximilian Hils
0929e74b4e fix compat with Python 3.5.0 2016-12-28 14:20:53 +01:00
Maximilian Hils
c78ffbf16d fix url.parse tests for Python 3.6 
This is a simpler version of @Kriechi's patch.
 2016-12-27 19:09:56 +01:00
Thomas Kriechbaumer
24751965f9 remove output of dumper during tests 2016-12-21 23:22:14 +01:00
Thomas Kriechbaumer
a196493a7a remove output of webmaster during tests 2016-12-21 23:13:53 +01:00
Maximilian Hils
77cd9224f9 Merge pull request #1880 from mhils/1877 
Fix #1877
 2016-12-19 19:04:45 +01:00
Maximilian Hils
4f3b50e417 fix io compat with previous releases 2016-12-19 16:19:37 +01:00
Maximilian Hils
fbce37054f fix #1877 2016-12-19 16:10:41 +01:00
Maximilian Hils
c4e9000021 fix #1858 2016-12-19 01:15:10 +01:00
Aldo Cortesi
cf15a3c3ef Merge pull request #1862 from mhils/remove-intercept-events 
Remove intercept events, update view manually
 2016-12-17 09:15:11 +13:00
Maximilian Hils
388fa7e716 remove intercept events, update view manually 2016-12-16 05:43:38 +01:00
Aldo Cortesi
e387c68b38 console: mark view indiciator, fix empty flow list behaviour 
Fixes #1782
 2016-12-16 11:57:45 +13:00
Aldo Cortesi
6b5673e849 Merge pull request #1845 from mhils/mitmweb-improvements 
Mitmweb Improvements
 2016-12-16 10:04:12 +13:00
Maximilian Hils
727abdba44 Merge pull request #1852 from mhils/simplenamespace 
Use types.SimpleNamespace instead of custom NS class
 2016-12-13 15:06:14 +01:00
Maximilian Hils
44f94c8844 use types.SimpleNamespace instead of custom NS class 2016-12-13 14:23:21 +01:00
Maximilian Hils
e5b3c8bed3 fix #1850 2016-12-13 14:12:18 +01:00
Maximilian Hils
d854e08653 [web] various fixes 2016-12-12 00:08:29 +01:00
Aldo Cortesi
a99bf0814c console: clean path prompt handling 
- Don't trigger callback if no path is specified
- Expand userdir centrally
- Catch IO errors from callbacks and turn them into status messages
 2016-12-12 09:13:43 +13:00
Aldo Cortesi
ca33bea296 Extend type checker validate Sequence specs 2016-12-12 07:39:59 +13:00
Maximilian Hils
3e37cbd061 minor fixes 2016-12-10 12:06:33 +01:00
Maximilian Hils
123ef043dc add new xml/html pretty-printer 🎉 2016-12-10 11:40:07 +01:00
Maximilian Hils
293b79af91 remove lxml-dependent code 2016-12-10 10:19:05 +01:00
Maximilian Hils
a7ba2f7b46 update tests, increase coverage, add type info 2016-12-09 21:26:02 +01:00
Aldo Cortesi
e943147fc3 options: add a merge method to assimilate options 
Fixes #953
 2016-12-08 16:12:40 +13:00
Aldo Cortesi
b2695dbc6a Minor tweaks 2016-12-08 10:21:06 +13:00
Aldo Cortesi
a617e3b5f7 Amalgamate all the Options objects 2016-12-08 10:21:06 +13:00
Aldo Cortesi
d742d4fb8c Replace configargparse with argparse, enable new options loading 
This commit ditches configargparse and enables the new config file format. The
default location is ~/.mitmproxy/config.yaml. Unifying all of the various
Options objects will follow in the next patch.
 2016-12-08 10:21:06 +13:00
Aldo Cortesi
150372e297 options: defaults are a class attribute 2016-12-08 10:21:06 +13:00
Aldo Cortesi
00c897a185 options: save defaults, add .reset() to restore defaults 
Use .reset() in console app to clear options.
 2016-12-08 10:21:06 +13:00
Aldo Cortesi
d74cac265a Add YAML serialization of options 
This uses ruamel.yaml. The library seems well-supported, and can do in-place
modification of config files that retains user comments and file structure.
 2016-12-08 10:21:06 +13:00
Aldo Cortesi
c94cd512d1 options: defaults are a class attribute 2016-12-08 10:21:06 +13:00
Aldo Cortesi
c062e302e9 Add OptManager.subscribe, use it to clean up palettes in console 
.subscribe lets you subscribe a function to a specified set of options.
 2016-12-08 10:21:06 +13:00
Aldo Cortesi
0c091bd92b Options - avoid mutation, API cleanup, has_changed 
- Always return a deepcopy of options to avoid accidental mutation of options
state.
- Remove .get(opt, default). This is an inappropriate API for Options - trying
to retrieve an option that doesn't exist should always be an error.
- Add the has_changed method that checks if an option differs from the default,
use it in mitmproxy console.
 2016-12-08 10:21:06 +13:00
Aldo Cortesi
b231836c70 Get rid of tfile testing option 
It's weird, it's ugly, it's getting in the way of my options refactoring, and
it must therefore die.
 2016-12-08 10:21:05 +13:00
Aldo Cortesi
297493801d Clean up dump tests 
- Remove tests that redundantly test addon funtionality that we've already
tested more comprehensively elsewhere.
- Extend to 100% coverage for tools/dump.py
 2016-12-08 10:21:05 +13:00
Aldo Cortesi
b4d33aaebf options: save defaults, add .reset() to restore defaults 
Use .reset() in console app to clear options.
 2016-12-08 10:21:05 +13:00
Thomas Kriechbaumer
47196e8676 tls_parser: allow optional extensions 
fixes #1816
 2016-12-06 19:51:57 +01:00
Thomas Kriechbaumer
1a36efbb6a simplify ALPN and OpenSSL on macOS 2016-12-04 11:04:06 +01:00
Thomas Kriechbaumer
cc6aa1f542 websocket: update close handshake tests 2016-11-29 23:47:19 +01:00
Maximilian Hils
34bf3a2496 fix #1783 (#1794) 2016-11-28 13:29:06 +01:00
Maximilian Hils
3154dc87fd fix pyinstaller 
amongst other things, fix #1746
 2016-11-28 00:51:52 +01:00
Maximilian Hils
568f40c810 Fix transparent mode initialisation, refactor mitmproxy.platform (#1787) 2016-11-25 17:32:23 +01:00
Maximilian Hils
222106916e Merge branch 'mitmweb-eventlog' 2016-11-23 22:47:50 +01:00
Maximilian Hils
45332006a3 mitmweb: 100% app test coverage, numerous fixes 2016-11-23 22:35:07 +01:00
Thomas Kriechbaumer
d3bd04dec0 disable TCP support for now 2016-11-23 10:18:45 +01:00
Thomas Kriechbaumer
5dfc199086 websocket: add tests 2016-11-23 10:18:45 +01:00
Thomas Kriechbaumer
4beb693c9c websocket: change cmdline option 2016-11-23 10:18:45 +01:00
Thomas Kriechbaumer
ffb3988dc9 rename WebSocket{s,} protocol 2016-11-23 10:18:45 +01:00
Maximilian Hils
f1662cbfd7 add eventstore tests 2016-11-23 01:27:51 +01:00
Yoginski
ebff5f2466 Descriptive error message for SSL context initialization failure (#1767) 2016-11-21 09:40:09 +01:00
Maximilian Hils
9af8f4bb31 organize examples 
This commit is largely based on work by Thiago Arrais (@thiagoarrais)
and Shane Bradfield (@l33tLumberjack). I wasn't really able to get their
PR reasonably merged onto the latest master, so I reapplied their changes
manually here and did some further improvements on that.
 2016-11-21 02:28:10 +01:00
Maximilian Hils
f74e561524 rewrite absolute-form HTTP requests to relative form, fix #1759 (#1765) 2016-11-20 17:21:21 +01:00
Thomas Kriechbaumer
f45034e8f1 tests: fix race condition 2016-11-20 17:02:26 +01:00
Maximilian Hils
5ce370e2a9 lint! 2016-11-18 12:59:59 +01:00
Aldo Cortesi
8065b44eed addons.view: clarify modification events 
This adds a set of store modification events, and uses them for flow settings.
This addresses a bug where settings could persist even after flows were deleted.
 2016-11-15 09:53:17 +13:00
Aldo Cortesi
9b08279c7c addons.proxyauth: out with the old, in with the new 
- Strip out old auth mechanisms, and enable addon
- Disable web app auth for now - this should just use the Tornado auth stuff
 2016-11-13 19:59:59 +13:00
Aldo Cortesi
dc88b7d110 addons.proxyauth: complete and test 2016-11-13 18:14:23 +13:00
Aldo Cortesi
e644d2167c stub out proxyauth addon 
Stub out basic workings, add and test configure event.
 2016-11-13 16:22:33 +13:00
Aldo Cortesi
fe01b1435a upstream_proxy_auth -> upstream_auth 
Also clarify what this does in commandline help.
 2016-11-13 11:50:28 +13:00
Aldo Cortesi
3b00bc339d Complete upstream authentication module 
- Handles upstream CONNECT and regular requests, plus HTTP Basic for reverse
proxy
- Add some tests to make sure we can rely on the .via attribute on server
connections.
 2016-11-13 11:43:27 +13:00
Aldo Cortesi
a9b4560187 Refine handling of HTTP CONNECT 
- CONNECT requests do not generate the usual http events. Instead, they
generate the http_connect event and handlers then have the option of setting an
error response to abort the connect.
- The connect handler is called for both upstream proxy and regular proxy CONNECTs.
 2016-11-12 18:28:37 +13:00
Aldo Cortesi
38f8d9e541 Add the http_connect event for HTTP CONNECT requests 2016-11-12 12:44:43 +13:00
Aldo Cortesi
bc01a146b0 Upstream proxy auth to addon 2016-11-12 11:58:04 +13:00
Aldo Cortesi
3958940420 Test failure during 100-continue 
Also:

- Remove duplicate and unused code
- Tighten scope of HttpReadDisconnect handler - we only want to ignore this for
the initial read, not for the entire block that includes things like the expect
handling.
 2016-11-12 11:58:04 +13:00
Aldo Cortesi
82ac7d05a6 Bug: ask requestheaders before request body is read 
Also add the beginnings of a test suite to exercise issues like this.
 2016-11-12 11:58:04 +13:00
Thomas Kriechbaumer
184e29e119 http2: add push-promise metadata test 2016-11-05 05:13:36 +01:00
Aldo Cortesi
067198a5dd Remove deprecated interfaces 2016-11-04 11:47:47 +13:00
Aldo Cortesi
6c8c4465d9 script.concurrent: tests to taddons 2016-11-04 10:59:41 +13:00
Aldo Cortesi
4f4db223fe addons: convert last of the test suite to taddons 2016-11-04 09:17:56 +13:00
Aldo Cortesi
deb66d3cac addons.setheaders: fix configure bug, tests to taddons 2016-11-04 09:09:39 +13:00
Aldo Cortesi
b51a96081a addons.view: test suite to 100% 
This takes coverage for all of /addons to 100%
 2016-11-04 08:55:01 +13:00
Aldo Cortesi
17e828b243 addons.view: test improvements 2016-11-03 20:53:06 +13:00
Aldo Cortesi
5ade93f2ad addons.streambodies: 100% test coverage 2016-11-03 20:16:07 +13:00
Aldo Cortesi
d7d6edb3d1 addons.serverplayback: 100% test coverage 2016-11-03 20:05:21 +13:00
Aldo Cortesi
e1fc80937d addons.serverplayback: test suite to taddons 2016-11-03 19:43:33 +13:00
Aldo Cortesi
e9a96f4d7f addons.script: 100% test coverage 2016-11-03 19:36:34 +13:00
Aldo Cortesi
d9538637c3 addons.script: convert to test.taddons 2016-11-03 18:57:34 +13:00
Aldo Cortesi
d31f2698a5 addons.onboarding: remove duplicate code 2016-11-03 16:20:38 +13:00
Aldo Cortesi
9fc6674151 addons.intercept: 100% test coverage 2016-11-03 16:16:29 +13:00
Aldo Cortesi
77e6dfe35c streamfile: make options less crazy 
Instead of a (file, mode) tuple, move to "streamfile" as the path, and
"streamfile_append" to specify if we should append or not.

Leave the command-line options intact for now.
 2016-11-03 15:18:55 +13:00
Aldo Cortesi
9f77c79227 FileStreamer -> StreamFile 
options.wfile -> options.streamfile
 2016-11-03 15:00:18 +13:00
Aldo Cortesi
43a8221989 addons: filestreamer test cov to 100% 
Also address a bug in taddons that caused options changes that raised errors to
be retained rather than rolled back.
 2016-11-03 14:50:23 +13:00
Aldo Cortesi
fbdce4b629 addons filstreamer: tests and bugfixes 
- Move tests to taddons
- Fix a bug where options updates caused streaming to halt
 2016-11-03 12:44:19 +13:00
Aldo Cortesi
5109fd8ecb Change antiquated test cipher suites 
This breaks Travis, possibly due to OpenSSL version.
 2016-11-03 08:23:19 +13:00
Aldo Cortesi
b867fb35a3 addons: dumper spit and polish 
- 100% test coverage
- Cleanups
- Add test/mitmproxy/addons/dumperview.py, a small utility for viewing dumper
output variations
 2016-11-02 22:47:49 +13:00
Aldo Cortesi
a75b3474a4 tests: convert test_dumper to use taddons 2016-11-02 11:17:08 +13:00
Aldo Cortesi
59b269425f tests: client_playback coverage 2016-11-02 10:59:33 +13:00
Aldo Cortesi
7e6d014f8f tests: pathod/tutils.py -> pathod/tservers.py 
And remove all aliases for mitmproxy.test.tutils
 2016-11-02 10:06:25 +13:00
Aldo Cortesi
c55e8d8f62 tests: clean up test/mitmproxy/tutils.py 
Remove aliases, use mitmproxy/test/tutils instead.
 2016-11-02 09:44:18 +13:00
Aldo Cortesi
fc7606bd98 tests: minor cleanup for tests/mitmproxy/tutils.py 2016-11-02 08:51:26 +13:00
Aldo Cortesi
83fdd82a52 addons anticom, anticache: expand tests, new truss 2016-11-01 17:38:45 +13:00
Aldo Cortesi
50deaf56bf Remove addons.State 
- Do a first-pass port of mitmweb. Some stuff still broken.
- Remove more all State tests from the test suite
 2016-11-01 10:06:28 +13:00
Aldo Cortesi
9d9735dd07 Replace addons.state in test suite 
Use a super-simple flow recorder instead.
 2016-11-01 09:36:24 +13:00
Aldo Cortesi
daf355bb4c console: add caching sort keys 
This is a tad complicated. The underlying sorted list implementation expects
the sort key to be stable for the lifetime of the object. However, if we sort
by size, for instance, the sort order changes as the flow progresses through
its lifecycle. We address this through two means:

- Let order keys cache the sort value by flow ID.
- Add a facility to refresh items in the list by removing and re-adding them
when they are updated.
 2016-10-30 21:11:30 +13:00
Aldo Cortesi
9abfb1aac2 console: re-add follow focus 2016-10-30 19:47:32 +13:00
Aldo Cortesi
f08b57fb9b console: "v" to reverse flow order 2016-10-30 17:57:43 +13:00
Aldo Cortesi
a67a591893 console: o for flow order 
"o" shortcut and --order cmdline option to change flow order.

Options is now bound to "O".
 2016-10-30 17:42:34 +13:00
Aldo Cortesi
6b6c44551a addons.view: Add first-class support for marking 
Marking now works differently - it's no longer just another filter, it's
applied in addition to the filter. This means you can apply a filter, mark some
flows, and then toggle between marked and unmarked flows matching the filter.

I'm leaving the filter for marked flows intact - it will come in handy in other
situations.
 2016-10-30 15:45:36 +13:00
Aldo Cortesi
2b76db1272 console: Port from state to view 
First phase of the port - basic flow list and flow view functionality working.
More to come.
 2016-10-30 14:56:36 +13:00
Aldo Cortesi
b9eb1a3479 Add options.default_contentview 
This is the first step in a longer process of revamping content views. For the
moment, the option is not exposed on the command line.
 2016-10-30 12:05:37 +13:00
Aldo Cortesi
944dcbaaa0 Cleanups: remove intercept from addons.state, unused imports 2016-10-30 11:13:05 +13:00
Aldo Cortesi
81a00f6f76 Replace interception in mitmproxy and mitmweb with addon 
Fixes #1674
 2016-10-30 11:07:09 +13:00
Aldo Cortesi
d1f14961ed test.taddon: Add cycle() method, use it to test addons.intercept 2016-10-30 10:52:57 +13:00
Aldo Cortesi
b4904d33ba Move test flow creators to mitmproxy.test.tflow 2016-10-30 10:12:39 +13:00
Aldo Cortesi
2dc3284fbb Add addons.intercept 
- Add an addon to handle intercept based on a filter pattern
- Start sketching out a nicer testing truss for addons in mitmproxy.test.taddon
 2016-10-29 15:44:48 +13:00
Aldo Cortesi
71d2636594 Sync 2016-10-29 12:35:10 +13:00
Aldo Cortesi
005c22445b addons.view: focus.focusflow -> focus.flow 2016-10-29 11:55:44 +13:00
Aldo Cortesi
7ecaeb0214 addons.view.focus: next and prev methods 2016-10-29 11:50:10 +13:00
Aldo Cortesi
32a0a7b860 addons.view: flow settings 
Add a flow settings mechanism, enable focus and settings unilaterally.
 2016-10-29 11:37:32 +13:00
Aldo Cortesi
14df969434 addons.view.focus: Better handling of view refresh 
When we refresh and our current focus goes out of scope, we set the focus to
the element nearest the old focus.
 2016-10-29 11:08:35 +13:00
Aldo Cortesi
90e7142b5c addons.View: better order reversal 
Deal with some subtleties in order reversal, add a toggle method that emits
refresh.
 2016-10-29 10:55:57 +13:00
Aldo Cortesi
12a70d03ad addons.view: Add a focus tracker 2016-10-29 10:17:29 +13:00
Aldo Cortesi
9dcc3a3e20 addons.View: hook up signals 2016-10-29 08:25:19 +13:00
Aldo Cortesi
69bacee1d8 Sketch out addons.View 
The first iteration of a replacement for addons.State
 2016-10-29 08:25:19 +13:00
Thomas Kriechbaumer
44ac370f08 upgrade h2 to >=2.5.0 2016-10-27 15:10:53 -07:00
Maximilian Hils
97f1236c99 Merge pull request #1670 from chhsiao90/display-cipher-#582 
Resolved #582: display ClientConnection select cipher of TLS
 2016-10-27 11:38:22 -07:00
chhsiao90
d52f35428c Show tls version on console 2016-10-26 14:36:14 +08:00
Maximilian Hils
b1bdae3d1c typecheck options, fix current inconsistencies 2016-10-25 20:45:48 -07:00
chhsiao90
960f2e8bf0 Resolved #582: display ClientConnection select cipher of TLS 2016-10-26 11:32:42 +08:00
Maximilian Hils
145c2892f7 Merge pull request #1664 from chhsiao90/sni-display-#1639 
Resolved #1639: display sni on ClientConnection
 2016-10-24 19:56:05 -07:00
chhsiao90
39ac29e37c Resolved #1639: display sni on ClientConnection 2016-10-25 10:46:53 +08:00
Maximilian Hils
ef4e9b2b85 Merge pull request #1656 from mhils/improve-export-2 
Improve Flow Export
 2016-10-24 19:19:58 -07:00
Sahil Chelaramani
7b3505336a replace mitmproxy.utils.lrucache with functools.lru_cache 2016-10-24 16:03:16 -07:00
Maximilian Hils
e87daa70f3 improve flow export 2016-10-22 20:32:39 -07:00
Maximilian Hils
ea2d6474bf Merge pull request #1649 from cortesi/kill 
Don't call error for flow kill
 2016-10-22 19:22:50 -07:00
Maximilian Hils
a1a792aeac various encoding fixes, fix #1650 2016-10-22 18:47:12 -07:00
Aldo Cortesi
6170493615 Don't call error for flow kill 
This is now the error handler on master, so whatever the intetion was it's now
definitely wrong.
 2016-10-22 15:56:42 +13:00
Aldo Cortesi
cc8b422d9d multidict: remove to_dict 
We never use it, and it is dangerously ambiguous when a key is associated with
a list.
 2016-10-21 11:42:21 +13:00
Aldo Cortesi
18ee6255c0 multidict: ditch ImmutableMultiDict 
A contorted class we only use for cookie attributes. We don't need it.
 2016-10-21 11:40:05 +13:00