Grasscutters/mitmproxy
3
0
Fork 0
mirror of https://github.com/Grasscutters/mitmproxy.git synced 2024-11-29 19:08:44 +00:00
Code Issues Projects Releases Wiki Activity
7,123 Commits 1 Branch 67 Tags 50 MiB
f31d5dc3eb
Commit Graph

1621 Commits

Author SHA1 Message Date
Aldo Cortesi
217addbf31 commands: view.go 
bind G to "view.go -1"
bind g to "view.go 0"
 2017-04-29 08:40:38 +12:00
Aldo Cortesi
0b090f7ae1 Commands, core update event 
This patch:

- Introduces a core update() event that should be invoked whenever flows are
changed outside of the normal lifecycle.
- Extend view.resolve to know about @all, which matches all flows in the view.
- Add a core flow.resume comand, which resumes flows and broadcasts an update event.
- Define flow list bindings for:

A -> flow.resume @all
a -> flow.resume @focus
d -> view.remove @focus
z -> view.remove @all
 2017-04-29 08:01:00 +12:00
Charles d'Hondt
65202f5f1c Added ldapsss vef 2017-04-28 17:00:21 +02:00
Charles d'Hondt
f91ed91bf1 fix 2017-04-28 16:40:34 +02:00
Charles d'Hondt
5fb18ad275 Added LDAP Auth 2017-04-28 16:19:27 +02:00
Maximilian Hils
769123498c fix mitmweb's content security policy 2017-04-28 14:14:56 +02:00
Aldo Cortesi
6af1a49464 commands: add a command.command decorator 
Use this for our built-ins and the console commands.
 2017-04-28 15:07:52 +12:00
Aldo Cortesi
7ff84673fd Support unary flow command arguments 
Use this to add a console.view.flow command and bind "enter" in flowlist.
 2017-04-28 14:05:35 +12:00
Aldo Cortesi
0ff0253b3e console: save flows to command 2017-04-28 12:15:28 +12:00
Aldo Cortesi
cfae95f5c3 console: start migrating hotkeys to keymap 
This shifts a set of flow list keys to keymaps. Much more to come.
 2017-04-28 12:10:48 +12:00
Aldo Cortesi
ce01cb9c09 command: partials 
Support partial commands in console.command, use it to keybind "i" for
intercept, which prompts the user with ":set intercept=".
 2017-04-28 11:24:28 +12:00
Aldo Cortesi
8a07059cf4 commands: add the core command addon, and the command "set" 
The set command sets an option using the same syntax as commandline --set.
 2017-04-28 10:41:44 +12:00
Aldo Cortesi
be1b76b975 console: add a keymap 
This sketches out a keymap system for consone, and adds the first few top-level
commands and mappings.
 2017-04-28 10:21:15 +12:00
Aldo Cortesi
18edc11145 console: interactive command browser 
Simple browser that lets users view and select commands interactively.
Key binding for this is still to be sorted out.
 2017-04-28 07:43:58 +12:00
Aldo Cortesi
b73b06b364 Merge pull request #2274 from cortesi/alerts 
Add an "alert" log level.
 2017-04-28 06:53:51 +12:00
Aldo Cortesi
aab6bf747c Merge pull request #2273 from cortesi/commands 
Commands
 2017-04-28 06:53:41 +12:00
Thomas Kriechbaumer
adce1a8235 Merge pull request #2277 from Kriechi/bump-cryptography 
bump cryptography min-version
 2017-04-27 20:11:15 +02:00
Thomas Kriechbaumer
3e82f4d6c4 bump cryptography min-version 
This removes OpenSSL v0.9.8 support.
 2017-04-27 19:39:56 +02:00
cle1000
115a31c9ba change import of prop-types 2017-04-27 10:54:21 +02:00
Aldo Cortesi
218e127e74 Add an "alert" log level. 
This has the same urgency as "info", but also signals to interactive tools that
the user's attention should be drawn to the output, even if they're not looking
at the event log. In the console app, this means the message appears in the
status bar with a timeout.
 2017-04-27 19:08:01 +12:00
Aldo Cortesi
97000aa85c command: save.file flowspec path -> None 
Our first user-facing command. The following commands do the obvious things:

save.file @marked /tmp/flows
save.file @focus /tmp/flows
save.file @hidden /tmp/flows
save.file "~m get" /tmp/flows
 2017-04-27 17:05:00 +12:00
Aldo Cortesi
b7afcb5dc2 addons.streamfile -> addons.save 
Options:

    streamfile -> save_stream_file
    streamfile_filter -> save_stream_filter
 2017-04-27 15:58:54 +12:00
Aldo Cortesi
8c4810f606 console: flow resolution command 
This is our first built-in command, which will be used by very many other
commands.

Also add a --commands option to dump all commands, analogous to --options.
 2017-04-27 15:27:51 +12:00
Aldo Cortesi
ee3dd3f3c5 console: very basic interactive command executor 2017-04-27 12:00:41 +12:00
Aldo Cortesi
169baabcab Basic outline of the command subsystem 
- Add a command manager
- Sketch out the type system with a few simple supported types
 2017-04-27 11:09:40 +12:00
Aldo Cortesi
2a46f3851a Merge pull request #2265 from cortesi/addons 
Addons and addon testing
 2017-04-27 07:40:14 +12:00
Maximilian Hils
0f9081b18d Merge pull request #2269 from mhils/issue-2250 
fix #2250, add type info to cookie module
 2017-04-26 14:42:48 +02:00
Maximilian Hils
87610cc8b2 fix #2250, add type info to cookie module 2017-04-26 14:17:14 +02:00
Maximilian Hils
d5ea08db62 Merge pull request #2258 from mhils/readfile 
Integrate readstdin into readfile
 2017-04-26 13:40:51 +02:00
Maximilian Hils
0a8e54edea Merge pull request #2268 from mhils/issue-2257 
fix #2257
 2017-04-26 13:39:19 +02:00
Maximilian Hils
1aa6d9d324 fix #2257 2017-04-26 12:26:49 +02:00
Maximilian Hils
ca2827886a separate reading from stdin into its own addon 2017-04-26 12:11:37 +02:00
Maximilian Hils
df7701bb6d fix #2228 2017-04-26 11:32:24 +02:00
Maximilian Hils
b3a1143338 integrate readstdin into readfile addon 2017-04-26 10:56:32 +02:00
Maximilian Hils
2b500f234f typecheck: add support for typing.Any 2017-04-26 10:54:36 +02:00
Aldo Cortesi
5327756377 Addons and addon testing 
- Fix some loading sequence bugs affecting command-line script invocation
- Allow addons to over-ride existing options (with a warning). We need this for
reloading.
- Convert har_dump to new-style arguments, fix and re-instate its test suite.
- Covnert miscelaneous other exmples to new-style args.
 2017-04-26 19:56:33 +12:00
Aldo Cortesi
b72f139093 configure(options, updated) -> configure(updated) 
Options are now available globally on ctx, so the first argument of configure
is redundant.
 2017-04-26 11:01:27 +12:00
Aldo Cortesi
f90b4c2ff0 Move options into ctx 
Many addons currently save options on configure(), either as individual options
or sometimes by saving the entire options object. The current options should
simply be available on the ctx object, simplifying state management for addons
considerably.
 2017-04-26 10:25:56 +12:00
Aldo Cortesi
7aa2081894 Remove watchdog, solidify script testing 
- Remove the watchdog dependency. We now just stat the script file every 2
seconds to check for an updated mtime.
- Further solidify our script testing, and in particular make the example tests
nicer. These should exemplify how we want users to test their own addon
scripts. More work on addon testing to follow.
 2017-04-26 09:01:40 +12:00
Aldo Cortesi
e6eeab6094 Revamp how addons work 
- Addons now nest, which means that addons can manage addons. This has a number
of salutary effects - the scripts addon no longer has to poke into the global
addons list, we no longer have to replace/remove/boot-outof parent addons when
we load scripts, and this paves the way for making our top-level tools into
addons themselves.
- All addon calls are now wrapped in a safe execution environment where
exceptions are caught, and output to stdout/stderr are intercepted and turned
into logs.
- We no longer support script arguments in sys.argv - creating an option
properly is the only way to pass arguments. This means that all scripts are
always directly controllable from interctive tooling, and that arguments are
type-checked.

For now, I've disabled testing of the har dump example - it needs to be moved
to the new argument handling, and become a class addon. I'll address that in a
separate patch.
 2017-04-25 22:13:44 +12:00
Ujjwal Verma
73692b909b Fixed formatting in mitmdump 2017-04-22 11:24:22 +05:30
Thomas Kriechbaumer
83a428e0b7 manually fix imports 2017-04-16 15:51:12 +02:00
Thomas Kriechbaumer
dadb50946e update kaitai and recompile formats 2017-04-16 15:51:11 +02:00
Ujjwal Verma
742127ef7b Fixes #2109 2017-04-09 18:55:20 +05:30
Robert C Jensen
7365f18542 fixes ipv6 authority form parsing in CONNECT 2017-04-07 15:46:21 -04:00
Vinay Dargar
55ba6ed14d Replace phrase 'the the' in 2 places 
Replaced with the appropriate "in the" and "to the".
There are multiple other occurences of this but they are in upstream python modules.
 2017-04-02 21:37:38 +05:30
Nikhil Soni
70a0f87570 Override set_focus to check for index error 2017-03-29 00:14:37 +05:30
Thomas Kriechbaumer
20776cf2fa fix #2201 2017-03-25 16:41:53 +01:00
Aldo Cortesi
65f0885bd6 addon loader: add boot_into, which replaces returning from start() 
While we're here, expand test coverage for addonmanager to 100%, and promote to
individual coverage.
 2017-03-25 10:48:12 +13:00
Aldo Cortesi
d69a411303 addons: an addonmanager.Loader is now passed to the load() event 2017-03-25 10:48:12 +13:00
Aldo Cortesi
541c1e8b9f addons: start -> load throughout 2017-03-25 10:48:12 +13:00
Thomas Kriechbaumer
ef9f0e22ea bump h2 2017-03-24 21:27:14 +01:00
Aldo Cortesi
335861f490 Merge pull request #2207 from nikofil/scripts-redirect-stdout 
scripts: redirect stdout to ctx.log.warn
 2017-03-24 14:58:33 +13:00
Nikos Filippakis
abf291b0f9 scripts: redirect stdout to ctx.log.warn 
Redirect messages written to stdout in scripts to ctx.log.warn.
(closes #1530)

Signed-off-by: Nikos Filippakis <nikolaos.filippakis@cern.ch>
 2017-03-24 02:35:02 +01:00
Aldo Cortesi
439c113989 Merge pull request #2204 from cortesi/consoleopts 
console options: load and save options to and from file
 2017-03-24 10:05:37 +13:00
Aldo Cortesi
0d3e2c6adf optmanager: make serialization and loading more robust 
Also handle errors in console options manager.
 2017-03-24 09:37:17 +13:00
Aldo Cortesi
eb66456d16 console: read and write options to and from file 2017-03-23 11:28:08 +13:00
Aldo Cortesi
44c3a24f8e Don't pass malformed request objects to our error handler 2017-03-23 10:48:20 +13:00
Matthew Shao
ab2fcbef8d Fix#2184, mitmweb -n dispalys incorrect message. (#2187) 
* Fixes #2184, python code.

* Fixes #2184, JavaScript code.

* [web] Update Footer.jsx (#2184)
 2017-03-22 01:26:04 +01:00
Aldo Cortesi
95d9ec88ac tox: mypy checking for entire codebase 
Also fix a few linting errors.
 2017-03-22 10:29:46 +13:00
LuD1161
cd8ce3c88e Added the level definition that can be used 2017-03-20 14:23:57 +05:30
Aldo Cortesi
cacad8373b Make tnetstrings pass mypy 
Mypy doesn't support recursive types yet, so we can't properly express
TSerializable nested structures. For now, we just disable type checking in the
appropriate locations.

https://github.com/python/mypy/issues/731
 2017-03-20 13:09:24 +13:00
Aldo Cortesi
4ca78604af Factor out an io module 
Include tnetstring - we've made enough changes that this no longer belongs in
contrib.
 2017-03-20 12:50:09 +13:00
Aldo Cortesi
3a8da31835 mypy all of the codebase bar tnetstring 
In some places, this involved removing type declarations where our types were
terminally confused. The grideditor specifically needs a cleanup and
restructure.
 2017-03-20 12:38:08 +13:00
Aldo Cortesi
b98ce71770 Fix sliding window type specs 2017-03-20 12:21:50 +13:00
Aldo Cortesi
2153c337e0 addons.view: we no longer have a None order state 
This bumps coverage back to 100% for addons
 2017-03-20 09:12:09 +13:00
Aldo Cortesi
b19fefd173 options: default_contentview uses lowercase 2017-03-19 14:39:16 +13:00
Aldo Cortesi
04f1a4443d options: reraise when there's an error with a partial update 
Fixes #2174
 2017-03-19 14:36:23 +13:00
Aldo Cortesi
8d68196fd0 console: use overlay chooser to pick flow view mode 
More consistent, and fixes status prompt overflow issue.
 2017-03-19 14:13:54 +13:00
Aldo Cortesi
1b6bc874b5 console options: "d" to reset this option to default 2017-03-19 14:01:32 +13:00
Aldo Cortesi
43e7b9d68c options: choices for content views and tighten console order spec 
Also clean up some leftovers from filter option name change.
 2017-03-19 13:39:28 +13:00
Aldo Cortesi
22d7c31ea8 dumper, mitmdump: kill filtstr 
Dumper now also uses view_filter. Passing arguments to mitmdump is now
precisely equivalent to setting both filestream_filter and view_filter.
 2017-03-19 13:21:46 +13:00
Aldo Cortesi
475a4e3eb0 streamfile: add streamfile_filter and use it instead of filtstr 2017-03-19 13:08:26 +13:00
Aldo Cortesi
6f745b90a8 console options: fix bug that caused keys to be handled by wrong component 2017-03-19 13:05:27 +13:00
Aldo Cortesi
b1ef4bce30 options: filter -> view_filter 2017-03-19 13:02:51 +13:00
Aldo Cortesi
cb18c91f13 console options: help for overlays, improved layout for overlay grid editor 2017-03-19 12:14:42 +13:00
Aldo Cortesi
49b0a67eb9 console options: add an overlay grid editor for sequence options 2017-03-19 10:32:22 +13:00
Aldo Cortesi
4e24c95a61 optmanager: cope with bound methods in .subscribe 
Fixes #2122
 2017-03-19 10:32:22 +13:00
Aldo Cortesi
3f50d5fdbb console options: handle choices settings 
This implements a generic chooser overlay, and uses it to handle setting
options that have fixed choices. We'll use this overlay elsewhere too.
 2017-03-19 10:32:22 +13:00
Aldo Cortesi
fea08ef919 console options: in-place editor for simple strings and ints 2017-03-19 10:32:22 +13:00
Aldo Cortesi
3a949f35f8 console options: better defaults display, g/G/D keyboard shortcuts 2017-03-19 10:32:22 +13:00
Aldo Cortesi
d759150734 console: options/help pane switching, toggle bools with enter 2017-03-19 10:32:22 +13:00
Aldo Cortesi
8130b9880a options cleanups 
- Move default client ciphers - no need to have a 10-line string default.
- Add some style guidance on options help
- Be consistent about reflowing options help
 2017-03-19 10:32:22 +13:00
Aldo Cortesi
bc67cee687 console: sketch out look and feel of the new Options editor 2017-03-19 10:32:22 +13:00
Aldo Cortesi
39659c7528 Make mypy succeed with imports on master.py 
We get little benefit from our mypy QA checks at the moment, because we skip
imports. This patch is what's needed to make mypy succeed with imports on a
single file: master.py

It also updates mypy to the current version, and enables a QA check.

Mypy bugs I encountered:

dict.update with kwargs not supported:

https://github.com/python/mypy/issues/1031

property setters and getters must be adjacent:

https://github.com/python/mypy/issues/1465
 2017-03-19 10:32:22 +13:00
Aldo Cortesi
5192810ff6 Make mypy succeed with imports on master.py 
We get little benefit from our mypy QA checks at the moment, because we skip
imports. This patch is what's needed to make mypy succeed with imports on a
single file: master.py

It also updates mypy to the current version, and enables a QA check.

Mypy bugs I encountered:

dict.update with kwargs not supported:

https://github.com/python/mypy/issues/1031

property setters and getters must be adjacent:

https://github.com/python/mypy/issues/1465
 2017-03-17 08:13:47 +13:00
Aldo Cortesi
c5e0dc64b9 Rip out master handler mechanism 
All events are now handled by addons, and we no longer support any events on
master.
 2017-03-16 18:33:24 +13:00
Aldo Cortesi
1410cbb4b6 Remove test handlers by using taddons.RecordingMaster 
This also means expanding and tweaking the recording master API, which we
reflect through the current test suite
 2017-03-16 18:05:57 +13:00
Aldo Cortesi
85ddc5056b Move mitmproxy console handlers to addon 2017-03-16 18:05:57 +13:00
Aldo Cortesi
b6abe96202 Move dump handlers to addon 2017-03-16 18:05:57 +13:00
Aldo Cortesi
b275257553 Merge pull request #2167 from mhils/update-dependencies 
update mypy
 2017-03-16 18:05:35 +13:00
Aldo Cortesi
1e6c51c698 Merge pull request #2152 from cortesi/readfile 
Extract flow reading into addons
 2017-03-16 15:51:21 +13:00
Maximilian Hils
112233891d update mypy 2017-03-15 23:10:21 +01:00
Aldo Cortesi
228a22b3c0 Add a light-weight custom event system, use it for keepserving 
This patch implements the lightweight event system I propose in #2144, adds a
custom event "processing_complete" that is triggered after file read, client
replay and server replay, and introduces a KeepServing addon to handle this for
mitmdump.
 2017-03-16 11:07:00 +13:00
Aldo Cortesi
169068c7ec Clean up addonmanager interface 
Clarify the plethora of invocation methods we've sprouted, correct some usages
in the codebase.
 2017-03-16 08:50:29 +13:00
Maximilian Hils
65c4a3cf1d add Request.make 2017-03-15 14:06:00 +01:00
Aldo Cortesi
ef582333ff Extract flow reading into addons 
This patch moves the final pieces of master functionality into addons.

- Add a ReadFile addon to read from file
- Add a separate ReadStdin addon to read from stdin, only used by mitmdump
- Remove all methods that know about io and serialization from master.Master
 2017-03-15 13:44:48 +13:00
Aldo Cortesi
eba6d4359c Merge pull request #2098 from iharsh234/bug-2086-2 
Fix crash when duplicating flow
 2017-03-15 09:20:33 +13:00
Maximilian Hils
30797755fb stateobject: automatically change id when copying 2017-03-14 17:08:40 +01:00
Maximilian Hils
375680a3be add connection ids 2017-03-14 17:08:40 +01:00
Maximilian Hils
2d016775fe Merge pull request #2149 from mhils/multidictview-copy 
Add MultiDictView.copy()
 2017-03-14 14:38:38 +01:00
Maximilian Hils
0de8f46c6e Merge pull request #2145 from mhils/ne 
py3: __ne__ delegates to __eq__ by default
 2017-03-14 14:38:00 +01:00
dequis
f351d0a307 Match ~d and ~u filters against pretty_host too 
Changed the ~u filter in the console UI to match the behavior of
mitmweb, which only matches against pretty_url, never against url.
 2017-03-14 01:57:22 -03:00
Aldo Cortesi
1f37743549 Merge pull request #2142 from mhils/2140 
Fix #2140
 2017-03-14 14:24:53 +13:00
Maximilian Hils
961747c4bc Merge pull request #2120 from ujjwal96/replace-from-file 
Replace from file through replacement editor #1458
 2017-03-14 02:23:31 +01:00
Maximilian Hils
30fd285ec7 multidictviews should not be serializable 2017-03-14 02:20:24 +01:00
Maximilian Hils
de1de3f626 add MultiDictView.copy 2017-03-14 02:18:02 +01:00
Maximilian Hils
875ce8c9c1 py3: __ne__ delegates to __eq__ by default 2017-03-14 00:40:15 +01:00
Aldo Cortesi
1b301ad5bb Move running() in to .tick() method to make sure it's called consistently 2017-03-14 11:41:20 +13:00
Aldo Cortesi
b745428b5c Enable custom options in config files 
We also now ignore unknown options in config files by default, and print a
warning if verbosity is incremented.
 2017-03-14 09:40:33 +13:00
Aldo Cortesi
2832e790fd Move serialization methods out of Options 
Attributes on options share a namespace with options themselves. It's getting
too crowded on our Options object, so let's shift some obvious stuff into the
module.
 2017-03-14 08:34:25 +13:00
Aldo Cortesi
c24f7d8e12 Optmanager: handle unknown options in value sets 2017-03-14 08:32:19 +13:00
Aldo Cortesi
75f83ba380 Show custom addon options in --options output 2017-03-14 08:32:19 +13:00
Aldo Cortesi
0c6663d0d5 Enable custom options for addons 
- Add an options parameter to the start() event. This is to be used by addons
on startup to add custom options.
- Add a running() event that is called once the proxy is up and running.
- With the new paradigm we can't log during master __init__, so add a tiny
termstatus addon to print proxy status to terminal once we're running.
 2017-03-14 08:32:19 +13:00
Maximilian Hils
2c6dcac97f fix #2140 2017-03-13 18:37:11 +01:00
Maximilian Hils
517aef1557 decouple mitmproxy and file format version 2017-03-13 16:46:07 +01:00
Maximilian Hils
05e11547f5 Merge remote-tracking branch 'origin/master' into pr-2120 
Conflicts:
	test/mitmproxy/addons/test_replace.py
 2017-03-12 23:33:49 +01:00
Maximilian Hils
0f4b523868 replacements: error right away when file does not exist 2017-03-12 23:28:49 +01:00
Thomas Kriechbaumer
1b045d24bc nuke tutils.tmpdir, use pytest tmpdir 2017-03-12 22:55:22 +01:00
harsh vijay
a8f7c30aa0 Fix crash when duplicating flow 2017-03-13 02:23:52 +05:30
harsh vijay
9a88594684 Fix crash when duplicating flow (#2) 2017-03-13 01:55:50 +05:30
Maximilian Hils
4684617d2c minor fixes 2017-03-12 20:24:50 +01:00
Ujjwal Verma
7f5fc0fdbd Removed ReplaceBase 2017-03-12 22:10:49 +05:30
Maximilian Hils
39154e628e Merge pull request #2118 from lymanZerga11/patch-3 
Update proxyauth.py to add proxyauth metadata
 2017-03-11 03:04:38 +01:00
Thomas Kriechbaumer
e9746c5182 Merge pull request #2114 from mitmproxy/fix-websocket-serialization 
make websocket flows serializable
 2017-03-10 21:15:46 +01:00
Thomas Kriechbaumer
49e0f23848 use flow.id to link websocket + handshake flows 2017-03-10 19:38:50 +01:00
Maximilian Hils
f3df4855af fix type hints 2017-03-10 19:15:44 +01:00
Thomas Kriechbaumer
45bf1ff64d disable h2c prior knowledge connections 2017-03-10 11:37:25 +01:00
Thomas Kriechbaumer
b1dd86d7ae make websocket flows serializable 
fixes #2113
 2017-03-09 19:08:59 +01:00
Ujjwal Verma
25703f8c53 Fixed Replacement test 2017-03-09 23:33:11 +05:30
Ujjwal Verma
07a8c4987a Removed replace-from-file option.Made requested changes 2017-03-09 23:20:35 +05:30
Aldo Cortesi
53178f35be Send change event when options are reset 2017-03-09 08:53:36 +13:00
Maximilian Hils
b345f5d432 typing.Type -> type 
`typing.Type` should have been `typing.Type[Any]`,
which in turn is equivalent to `type`
 2017-03-08 15:17:45 +01:00
Maximilian Hils
63179d9751 core -> core_option_validation 
longer, but much clearer for devs who are unfamiliar with the codebase.
 2017-03-08 15:17:07 +01:00
Maximilian Hils
618e0c6265 add type hints 2017-03-08 15:10:06 +01:00
lymanZerga11
716306538f Update proxyauth.py 2017-03-08 13:59:43 +08:00
lymanZerga11
cf83cbf2df Update proxyauth.py 2017-03-08 11:34:59 +08:00
Maximilian Hils
f276c7a80d change add_option parameter order 
name, type, value matches natural language ($x is a bool with a default value of $y)
and also matches the python type annotation order of name: type = value
 2017-03-08 00:52:37 +01:00
Maximilian Hils
8707928b16 unify server spec parsing 2017-03-08 00:18:34 +01:00
Aldo Cortesi
f0d6237a96 Merge remote-tracking branch 'upstream/master' into options 2017-03-08 08:22:49 +13:00
Maximilian Hils
88d48e38af Revert "clearly indentify git-checkout as dev version" 
This temporarily reverts commit 3b4d8d6eca,
which broke the snapshot builds.
 2017-03-07 14:49:52 +01:00
Maximilian Hils
96b798615c Merge pull request #2106 from davidpshaw/python_3_to_master 
Updated for python 3
 2017-03-07 12:12:47 +01:00
Aldo Cortesi
98ec3b77fe Carefully re-add a small number of short flags 
The mechanism for booleans attaches the short flag to whatever the opposite of
the default is.
 2017-03-07 21:01:21 +13:00
Aldo Cortesi
aed780bf48 Change the way proxy authetication is specified 
We now have one option "proxyauth". If this is "any", we accept any
credentials, if it starts with an @ it's treated as a path to an htpasswd file,
if it is of the form username:password it's a single-user credential.
 2017-03-07 20:18:30 +13:00
Aldo Cortesi
b0ba765598 Add type and choices to options dump commets. 2017-03-07 19:29:08 +13:00
Aldo Cortesi
320d8848ab Options tweaks 
- Regularise presentation and wording
- Help is mandatory for all options
- Auto-generate wording to say that sequence options can be passed multiple
times on the command-line
 2017-03-07 15:23:46 +13:00
Aldo Cortesi
10db254791 Remove less commonly used command-line options 
We now have --set, so only options that really deserve it get a dedicated flag.
I'm inclined to strip this back even more. Feel free to argue that YOUR
favourite option deserves special treatment here.
 2017-03-07 14:46:28 +13:00
Aldo Cortesi
ac3b0d69cc Add the --set option to set options directly 
The --set option is a universal flag for setting options. Some examples:

Turn on a boolean:

mitmdump --set onboarding=false

Add a value to a sequence:

mitumdupm --set setheaders=/foo/bar/voing

Zero a sequence:

mitumdupm --set setheaders
 2017-03-07 14:27:50 +13:00
Aldo Cortesi
79f5883c2f Option spacing and coverage 2017-03-07 13:47:39 +13:00
Aldo Cortesi
99a6b0dbc1 Add --options that dumps annotated option defaults 2017-03-07 13:16:28 +13:00
Maximilian Hils
193c707061 Merge pull request #2104 from lymanZerga11/patch-2 
Fixes #2102
 2017-03-06 23:33:33 +01:00
Maximilian Hils
8b79892246 Merge pull request #2105 from Kriechi/dev-version 
clearly indentify git-checkout as dev version
 2017-03-06 23:32:59 +01:00
David Shaw
5d82068218 Updated for python 3 2017-03-06 17:30:40 -05:00
Aldo Cortesi
d13df40753 Fix lint 2017-03-07 10:52:28 +13:00
Aldo Cortesi
3d9c2233be Flatten commandline arg structure, extract common run func for tools 2017-03-07 10:51:24 +13:00
Aldo Cortesi
b51df9a0b1 Cleanups and test coverage 2017-03-07 10:28:15 +13:00
Aldo Cortesi
2312cf6fb0 Automate slurping up command-line options 
Now that options are completely regular, we can automate reading them out of
arguments, rather than listing them out by hand like savages.
 2017-03-07 08:49:19 +13:00
Thomas Kriechbaumer
3b4d8d6eca clearly indentify git-checkout as dev version 2017-03-06 18:40:43 +01:00
lymanZerga11
fd5079b8a6 Update flowlist.py 2017-03-06 23:19:39 +08:00
Aldo Cortesi
edfd62e42a Replacements and setheaders are always strings 
Instead of having two representations we have one canonical specification.
Fixing the editor in console is left ot a further patch.
 2017-03-06 20:58:51 +13:00
Aldo Cortesi
82163a1e68 Unify mode specification 
We now have:

    --mode regular (the default)
    --mode transparent
    --mode socks5
    --mode reverse:SPEC
    --mode upstream:SPEC

Where SPEC is a host specification.
 2017-03-06 20:30:49 +13:00
Aldo Cortesi
f5fb6972aa Options unification: certs 
- Regularise to Sequence[str]
- Move conversion and checking into proxy config object
 2017-03-06 15:36:52 +13:00
Aldo Cortesi
9b1f40da37 Options unification: streamfile 
We now have one option to control this. If the path is prefixed with a "+" we
append, otherwise we overwrite.
 2017-03-06 13:48:37 +13:00
Aldo Cortesi
71a830c836 Options unification: string choices 2017-03-06 13:20:17 +13:00
Aldo Cortesi
e70b46672c Options unification: simple string sequences 2017-03-06 11:56:11 +13:00
Aldo Cortesi
18a6b66ba4 Options unification: simple strings 
Move all simple string options to the new scheme. Also regularise some names.
 2017-03-06 11:39:19 +13:00
Aldo Cortesi
201c65960e Options unification: sizes 
Start dealing with corner cases:

- Sizes are always stored in options as strings
- Add a new core addon that's responsible for verifying settings that don't
belong to an addon
- Add a _processed scratch space on the Options object for processed core
values to be stored in. This is pretty dirty, but less dirty than re-parsing
values every time. We'll come up with something better down the track.
 2017-03-06 09:07:16 +13:00
Aldo Cortesi
45d18ac8cb Start unifying options and the command-line: ints 
Moves all integer options apart from a few tricky ones like verbosity over to
auto generation. Also add a metavar argument to parser generation to support
this.
 2017-03-05 20:45:55 +13:00
Aldo Cortesi
f15a628561 Start unifying options and the command-line: booleans 
This commit:

- Adds a help field to options
- Adds a function to generate parser definitions from options
- Uses this to migrate all boolean flags over to the new system
- Makes all booleans consistently follow the --foo/--not-foo convention

There are a number of things left to be done here:

- Argparse doesn't give us a nice way to format --foo --not-foo help. Click
does, and moving to click is a goal down the track.
- For now, we remove all short aliases. I want to re-evaluate these
systematically once we have the new structure in place.
 2017-03-05 20:16:33 +13:00
Aldo Cortesi
67381ae550 Revamp options 
- Options are now explicitly initialized with an add_option method
- We have one canonical Options class - ditch dump.Options
 2017-03-05 15:10:03 +13:00
Maximilian Hils
2a1ed7f3eb Merge pull request #2082 from tarnacious/master 
Call error handler and mark flow on HTTPException
 2017-03-04 15:12:29 +01:00
Maximilian Hils
1bafe73a94 update release docs 2017-03-04 11:50:40 +01:00
Maximilian Hils
f71c11559c Merge pull request #2081 from s4chin/add-pretty-host 
web: Fix #1888
 2017-03-04 11:48:05 +01:00
Sachin Kelkar
40943f5618 web: Fix #1888 2017-03-03 16:14:20 +05:30
Nikhil Soni
0081d9b828 Merge branch 'master' into on-issues 2017-03-03 12:58:44 +05:30
Nikhil Soni
2e90373e4b Updates help msg and docs for --keep-host-header 2017-03-03 12:34:20 +05:30
tarnacious
fc3848345b Call error handler and mark flow on HTTPException 
This allows scripts to handle HTTPExceptions such as "HTTP Body too
large" raised in mitmproxy/net/http/http1/read.py:131
 2017-03-01 16:35:54 +01:00
Thomas Kriechbaumer
a505221e1a http2: fix h2 header normalization test (#2080) 2017-03-01 12:46:42 +01:00
Matthew Shao
7b4e60a2e7 Fix #2071, remove client_connect.mitmcert when dumping flow into json. 2017-02-27 21:45:26 +08:00
Maximilian Hils
2b3093fa1c Merge pull request #2069 from JordanLoehr/iss2068 
Added console_ prefix to options in palettepicker to resolve issue #2068
 2017-02-27 05:37:26 +01:00
Jordan Loehr
6fd9146d0d Added console_ prefix to options in palettepicket to resolve issue #2068 2017-02-26 17:08:00 -07:00
Maximilian Hils
0fdf2c0f4b Merge pull request #2067 from whackashoe/fix-2065 
fix #2065 by setting correct content encoding for brotli
 2017-02-26 22:10:12 +01:00
Jett
18a8834209 fix #2065 by setting correct content encoding for brotli 2017-02-26 14:49:50 -06:00
Thomas Kriechbaumer
e0bc1109c0 do not return IP addresses in SSLCert.altnames 2017-02-26 20:54:06 +01:00
Thomas Kriechbaumer
6d228dfb9f certs: coverage++ 2017-02-26 20:54:04 +01:00
Thomas Kriechbaumer
9b6986ea87 connections: coverage++ (#2064) 2017-02-26 20:50:52 +01:00
Thomas Kriechbaumer
d17309eda8 flowfilter: coverage++ 2017-02-24 18:47:56 +01:00
Maximilian Hils
fab3a8dcf4 fix constant definition 2017-02-24 13:39:03 +01:00
Thomas Kriechbaumer
3e9125a3c1 nuke tcp.Address and add proper IPv6 support 2017-02-24 13:39:02 +01:00
Thomas Kriechbaumer
d0d11cec7b Merge pull request #2055 from Kriechi/http2-normalize-headers 
http2: normalize headers before sending
 2017-02-23 16:04:11 +01:00
Thomas Kriechbaumer
bbe08bb8d9 http2: normalize headers before sending 2017-02-23 12:53:40 +01:00
Nikhil Soni
14d8c91250 Adds --keep-host-header option (#2039) 2017-02-22 21:49:31 +05:30
Maximilian Hils
ba76dbc672 Merge pull request #2048 from ujjwal96/ipv6-addresses 
Absolute IPv6 addresses supported
 2017-02-22 13:17:53 +01:00
Thomas Kriechbaumer
786cd214d6 update compat 2017-02-21 20:57:54 +01:00
Maximilian Hils
6b2383a9d8 minor adjustments 2017-02-21 20:57:29 +01:00
Daniel Lenski
48b3d1af2f store generated cert for each flow 
fixes #1935
 2017-02-21 20:57:29 +01:00
Thomas Kriechbaumer
cf0bce77a9 lint! 2017-02-21 19:40:06 +01:00
Thomas Kriechbaumer
0b1cdc382c bump dev version 2017-02-21 19:17:33 +01:00
Maximilian Hils
2df2fc1f38 Merge pull request #2051 from MatthewShao/fix-#1928-@concurrent-in-OOP 
Fix #1928, @concurrent decorator in class
 2017-02-21 15:38:34 +01:00
Maximilian Hils
29c4a43e50 docs++ 2017-02-21 15:38:20 +01:00
Maximilian Hils
96256579e6 update mitmweb resources 2017-02-21 15:33:13 +01:00
Maximilian Hils
aa6b0f299e minor changes 2017-02-21 13:00:04 +01:00
Matthew Shao
f9714fbf3e Fix #1928, @concurrent decorator now works for class methods. 2017-02-21 11:59:50 +08:00
Ujjwal Verma
11134b669e Absolute IPv6 addresses supported 2017-02-20 21:02:43 +05:30
Thomas Kriechbaumer
050245e842 Merge pull request #2017 from lymanZerga11/patch-1 
Catch ValueErrors from url.parse()
 2017-02-20 11:48:40 +01:00
Thomas Kriechbaumer
51e08a5b85 Merge pull request #2041 from krsoninikhil/debug 
Adds PyInstaller build indicator to --version argument
 2017-02-20 10:29:15 +01:00
Thomas Kriechbaumer
36352c9539 protobuf: coverage++ 2017-02-19 14:29:09 +01:00
Thomas Kriechbaumer
83c2de8849 http2: disable priority forwarding 2017-02-18 14:30:08 +01:00
Maximilian Hils
4158a1ae55 Merge pull request #2040 from mhils/request-host-header 
Add "Request.host_header"
 2017-02-18 12:08:54 +01:00
Thomas Kriechbaumer
8cbd6dca9f Merge pull request #2035 from Kriechi/fix-1916 
fix #1916
 2017-02-18 11:40:56 +01:00
Nikhil Soni
2a461d6b39 Adds PyInstaller build indicator to --version argument 2017-02-18 06:27:06 +05:30
Maximilian Hils
b9e31f213f .headers["host"] -> .host_header 2017-02-18 00:13:14 +01:00
Maximilian Hils
49c8e19f80 add request.host_header, fix #2036 2017-02-17 23:31:53 +01:00
Thomas Kriechbaumer
c04d14e53c fix #1916 2017-02-17 22:45:31 +01:00
mame82
ffdbccd571 Use existing urlencoding conventions when re-encoding edited form #1946 (#2022) 2017-02-16 15:03:27 +01:00
Maximilian Hils
d1186eea18 Merge pull request #2029 from mhils/proxyauth 
Improve ProxyAuth
 2017-02-15 16:52:20 +01:00
Maximilian Hils
2955e3d566 improve make_error_response 2017-02-15 15:55:08 +01:00
Maximilian Hils
4bac850bb1 fix #1722, fix #1734, refs #2019 2017-02-15 14:20:46 +01:00
Maximilian Hils
0d9c7ce50c image view: add fallback, catch all images but svgs 2017-02-15 14:04:24 +01:00
Thomas Kriechbaumer
8a6f8bd461 nuke Pillow 2017-02-15 00:54:14 +01:00
Matthew Shao
9b9d72594c fix websocket message display crash in console. 2017-02-14 21:21:39 +08:00
lymanZerga11
55e471af40 Catch ValueErrors from url.parse() 2017-02-12 23:06:11 +08:00
Maximilian Hils
3067a971f9 Merge pull request #2000 from s4chin/add-jpeg-parser 
Add jpeg parser
 2017-02-11 23:00:14 +01:00
Sachin Kelkar
245e24dcf3 Add sources of images, remove copyrighted image 2017-02-12 01:19:36 +05:30
Sachin Kelkar
ccca04b450 Fix coverage 2017-02-11 15:01:54 +05:30
Jihyun Yu
4adc575bad Add transparent proxy support for OpenBSD 
Add transparent proxy support for OpenBSD with pf divert-to.
 2017-02-11 14:05:22 +09:00
Sachin Kelkar
8eb1d34644 Add app1 marker and tests 2017-02-10 22:40:19 +05:30
Ujjwal Verma
809207195d closes #1828 script reloads on py file changes 2017-02-10 22:21:12 +05:30
Sachin Kelkar
c4e141a000 Add jpeg app0 sof0 com parsing 2017-02-10 11:16:23 +05:30
Ujjwal Verma
d6465b907f Closes #1580 Gives warning when explicit proxy configured in transparent mode (#1996) 2017-02-09 20:56:06 +01:00
Maximilian Hils
380ff50e57 Merge pull request #2003 from Kriechi/coverage++ 
test refactoring and coverage++
 2017-02-09 17:45:50 +01:00
Maximilian Hils
1084588103 Merge pull request #1998 from mhils/fix-change-upstream 
Fix change_upstream_proxy_server
 2017-02-09 16:39:20 +01:00
Thomas Kriechbaumer
2ff5d72236 minor changes 2017-02-09 16:08:36 +01:00
Thomas Kriechbaumer
0299bb5b2e eventsequence: coverage++ 2017-02-09 16:08:36 +01:00
Thomas Kriechbaumer
1847cf175c websockets, tcp, version: coverage++ 2017-02-09 16:08:35 +01:00
Thomas Kriechbaumer
2134b5b06a remove FIXME 2017-02-09 16:08:35 +01:00
Thomas Kriechbaumer
b5f0342664 remove verified remark 2017-02-09 16:08:35 +01:00
Maximilian Hils
d6d1ff0170 simplify state copy 2017-02-09 16:08:24 +01:00
Maximilian Hils
9b97b63891 fix change_upstream_proxy_server 2017-02-09 15:24:31 +01:00
Sachin Kelkar
5dd54ef012 Update kaitaistruct version to 0.6 2017-02-08 21:16:17 +05:30
Maximilian Hils
28c0596742 Merge pull request #1997 from s4chin/add-gif-parser 
Add gif parser and tests
 2017-02-08 12:09:48 +01:00
Sachin Kelkar
66ad95c330 Fix gif parsing error on some gifs and add test 2017-02-08 00:03:07 +05:30
Sachin Kelkar
4ef5de2cce Add gif parser and tests 2017-02-06 17:57:59 +05:30
Ujjwal Verma
c622622c59 Encoding fixes and tests 2017-02-05 18:59:01 +05:30
Maximilian Hils
d133b8baee Merge pull request #1989 from Kriechi/coverage++ 
coverage++
 2017-02-04 16:05:11 +01:00
Thomas Kriechbaumer
966ffaa3d6 coverage++ 2017-02-04 13:48:55 +01:00
Maximilian Hils
155670766e Merge pull request #1988 from krsoninikhil/ordereddict 
Closes #1977
 2017-02-04 13:46:44 +01:00
Nikhil Soni
e6de57ccc6 Changes view._store from dict to OrderedDict 2017-02-04 17:44:22 +05:30
Maximilian Hils
53f298ac41 Merge pull request #1967 from s4chin/remove-pillow 
Add png parser
 2017-02-03 18:58:34 +01:00
Maximilian Hils
bbdb7300fd Merge pull request #1983 from Kriechi/coverage++ 
coverage++
 2017-02-03 18:56:56 +01:00
Sachin Kelkar
15548ff433 Feedback 2017-02-03 18:40:50 +05:30
Thomas Kriechbaumer
bcaaa2f40b coverage++ 2017-02-03 11:55:33 +01:00
Sachin Kelkar
0674485e76 Fix and cleanup 2017-02-03 15:02:35 +05:30
Sachin Kelkar
88dbfd5257 Add iTXt and zTXt tests 2017-02-03 14:12:54 +05:30
Sachin Kelkar
abef020e07 Fix as per feedback and add more tests 2017-02-03 14:12:54 +05:30
Sachin Kelkar
6202958048 Add png parser 2017-02-03 14:12:54 +05:30
Ujjwal Verma
11b4b3209d Closes #1960 'Z' removes unmarked flows 2017-02-03 01:35:53 +05:30
Thomas Kriechbaumer
79aa994275 fix merge 2017-02-02 17:26:31 +01:00
Maximilian Hils
8b6e3d8bd3 Merge pull request #1979 from 0neblock/master 
Fixed following on flowlist
 2017-02-02 17:24:04 +01:00
Aidan Cyr
00a6551622 Fixed following on flowlist 
Fixed a focus_follow was missed when options were cleaned up.
 2017-02-02 23:00:47 +11:00
Thomas Kriechbaumer
ae008ed80b replace tutils.raises with pytest.raises + shim 2017-02-02 12:59:01 +01:00
Thomas Kriechbaumer
ec92d7f67e cleanup test utils 2017-02-02 12:59:01 +01:00
Thomas Kriechbaumer
380d8ec370 increase test coverage 2017-02-02 10:15:01 +01:00
Thomas Kriechbaumer
2606de25e4 fix linting error 2017-02-02 09:36:33 +01:00
lymanZerga11
4b10212caf Add except clause to catch script parsing errors (#1929) 2017-02-01 15:27:10 +01:00
Aldo Cortesi
914659e888 Regularise setheaders options 
As per replacements:

- Make the option type a string/tuple union
- Localise parsing strictly within the addon
- Adapt the console editor (we'll find a more elegant solution later)
 2017-02-01 12:22:05 +13:00
Aldo Cortesi
4cc75a9560 Revamp replacement hooks 
- Replacement specifiers can be either strings or tuples. This lets us cope
gracefully with command-line parsing (and posible quick interactive
specification) without having to special-case replacement hooks, or have
knowledge of hook specification leak outside the addon. We can also now use the
same command-line spec format in config files.
- Split replacement and replacement from file into separate addons and options.
Files are now read on each replacement, so you can edit replacement files in
place without restart.
- Modernise the test suite to use addon test helpers.

TODO: editing and displaying replace-from-file in console app
 2017-02-01 11:10:28 +13:00
Aldo Cortesi
02f51d043d Merge pull request #1965 from cortesi/optname 
Consolidate option names
 2017-02-01 08:39:35 +13:00
Maximilian Hils
45ace793d0 🙈 2017-01-30 19:30:35 +01:00
Aldo Cortesi
35aff3b783 Consolidate option names 
- Onboarding app options to onboarding*
- Console-specific options to console_*
- Web-specific options to web_*
 2017-01-30 11:15:12 +13:00
Thomas Kriechbaumer
3028e06fd2 simplify termlog outfile handling 2017-01-25 17:50:07 +01:00
nish21
ab45e4d183 write errors to stderr (#1952) 2017-01-25 17:49:56 +01:00
Thomas Kriechbaumer
acfd548fa2 Merge pull request #1956 from Kriechi/version-string 
improve version specifiers
 2017-01-23 22:53:02 +01:00
Maximilian Hils
2eaac31344 Merge pull request #1945 from Kriechi/ca-expired 
fix #939
 2017-01-23 21:22:38 +01:00
Thomas Kriechbaumer
63beaa18ce improve version specifiers 2017-01-23 20:33:12 +01:00
Thomas Kriechbaumer
5792e2c483 test disable_h2c_upgrade addon 2017-01-23 18:38:27 +01:00
Thomas Kriechbaumer
d674de298c test websocket addon 2017-01-23 15:45:53 +01:00
Thomas Kriechbaumer
006eb39cc5 fix #939 2017-01-23 15:14:56 +01:00
Maximilian Hils
2ca48e5d08 Merge pull request #1940 from s4chin/add-marked-filter 
web: Add ~marked filter
 2017-01-20 15:26:51 +01:00
Sachin Kelkar
bc8f5a2d71 web: Add ~marked filter 2017-01-20 09:17:09 +05:30
Thomas Kriechbaumer
72b753c60f provide git information with --version 
fixes #1848
 2017-01-19 20:27:29 +01:00
Maximilian Hils
fe43e629fd console: display http version in details 2017-01-10 15:26:11 +01:00
Maximilian Hils
d953d83773 mitmdump: fix reading from stdin 2017-01-10 10:37:28 +01:00
Maximilian Hils
88f3459c7d stronger tests, lint 2017-01-07 23:08:50 +01:00
Maximilian Hils
ede124a587 fix #1910 2017-01-07 23:08:50 +01:00
Maximilian Hils
c41bd3fafd minor legibility improvements 2017-01-07 23:08:50 +01:00
Maximilian Hils
b1ec7e78cd Revert "add strutils.replace_surrogates" 
This reverts commit 15ae88db08dbf2725c8c9f945f2006c4b2a10d46.
 2017-01-07 23:08:50 +01:00
Maximilian Hils
c21ee90deb add strutils.replace_surrogates 2017-01-07 23:08:50 +01:00
Maximilian Hils
042261266f minor encoding fixes 
- native() -> always_str()
  The old function name does not make sense on Python 3 only.
- Inline utility functions in message.py.
 2017-01-07 23:08:50 +01:00
Maximilian Hils
af194918cf fix HTTP reason phrase encoding 
While not explicitly spelled out in the RFCs,
the reason phrase is expected to be ISO-8859-1.
 2017-01-07 23:08:50 +01:00
lymanZerga11
3ea76a7f3e Controls the length of URL displayed using the -d option (#1903) 2017-01-05 14:37:20 +01:00
chhsiao90
af0539c526 Fixed edit form failed 2016-12-31 14:38:33 +08:00
chhsiao90
a5f1215eb2 Allow underscore in hostname 2016-12-30 21:03:22 +08:00
Maximilian Hils
eab360a02b fix IO type checking 2016-12-28 14:21:19 +01:00
Maximilian Hils
0929e74b4e fix compat with Python 3.5.0 2016-12-28 14:20:53 +01:00
Maximilian Hils
8185cf2724 update io compat for 2.0 2016-12-27 21:03:59 +01:00
Maximilian Hils
e531a97a8b bump version for next release cycle 2016-12-27 20:52:22 +01:00
Thomas Kriechbaumer
7221f49b25 Merge pull request #1887 from Kriechi/h2-enable 
enable HTTP/2 by default
 2016-12-27 20:49:09 +01:00
Maximilian Hils
6ae378aa20 bump version to 1.0.1 2016-12-27 19:26:28 +01:00
Thomas Kriechbaumer
51d57cfd4a py36: fix type information 2016-12-27 16:28:12 +01:00