Commit Graph

1621 Commits

Author SHA1 Message Date
Maximilian Hils
3209704086 fix addon tracebacks
`.tb_next` discards the first interesting frame,
this shouldn't happen.
2017-07-21 11:25:17 +02:00
Maximilian Hils
e1d0bc6de9 Merge pull request #2430 from MatthewShao/mitmweb-options
[web] [WIP] Mitmweb options
2017-07-20 18:01:53 +02:00
Maximilian Hils
8526ca9e17 Merge pull request #2454 from mhils/minor-improvements
Minor improvements
2017-07-20 17:45:40 +02:00
Maximilian Hils
7b0485d6d6 update mypy 2017-07-20 16:27:13 +02:00
Maximilian Hils
9176626dab fix type declaration 2017-07-20 15:43:36 +02:00
Maximilian Hils
b7bda8f4b2 shorten option help
We can have longer versions in the docs, but this makes the
options pages quite hard to parse.
2017-07-20 15:27:41 +02:00
Maximilian Hils
94d28831e1 replace deprecated use of SourceFileLoader.load_module 2017-07-20 15:23:22 +02:00
Maximilian Hils
779677bcc6 fix loading scripts with same filename 2017-07-20 15:21:09 +02:00
Matthew Shao
1f5ede2857 [web] Update error msg in PUT /options API. 2017-07-17 13:59:25 +02:00
Maximilian Hils
a81cd18d92 make eventsequence.iterate extendable 2017-07-15 12:26:38 +02:00
Maximilian Hils
af777f9b4f remove --shortversion
this has no benefit over --version and clutters our cmdline options.
2017-07-15 12:21:15 +02:00
Ujjwal Verma
fdc64be04a Allow remote addon 2017-07-15 10:01:14 +02:00
Thomas Kriechbaumer
3814f171dd Merge pull request #2427 from ujjwal96/protobuf
Kaitai parser for protobuf
2017-07-10 22:58:38 +02:00
Ujjwal Verma
3f269d2b68 Kaitai parser for protobuf 2017-07-10 01:24:43 +05:30
Maximilian Hils
062a58f848 Merge pull request #2423 from MatthewShao/mitmweb-options
[web] [WIP] Mitmweb options editor content
2017-07-05 16:25:38 +02:00
Matthew Shao
c1553c7602 [web] Broadcast options update in backend. 2017-07-05 08:42:41 +08:00
Matthew Shao
aa01a62df7 [web] Add keys argument for dump_dict in optmanager. 2017-07-05 08:40:21 +08:00
Ujjwal Verma
7e04ac0578 add websocket streaming option 2017-07-04 10:53:23 +02:00
Ujjwal Verma
9e1902be62 fix HTTP retry if sending a request fails once 2017-07-04 10:52:50 +02:00
Ujjwal Verma
389d37ebff websocket streaming 2017-07-04 10:52:50 +02:00
Ujjwal Verma
d4f35d7a4a request streaming for HTTP/2 2017-07-04 10:52:50 +02:00
Ujjwal Verma
47c9604aed request streaming for HTTP/1 2017-07-04 10:52:50 +02:00
Matthew Shao
d7bbfca167 [web] Change the response format of GET /options. 2017-07-02 12:17:16 +08:00
Ujjwal Verma
7dfa557441 Fixes #2315 2017-06-28 19:18:57 +05:30
Maximilian Hils
d58abc9200 Merge pull request #2402 from ujjwal96/request-replay
Fixes #2197
2017-06-27 11:18:12 +02:00
Ujjwal Verma
08735ab0ae request replay test 2017-06-26 00:53:14 +05:30
Thomas Kriechbaumer
646f26b0e2 Merge pull request #2412 from ujjwal96/ip-formatting
Few ip formatting fixes
2017-06-25 09:18:45 +02:00
Maximilian Hils
13dac9f212 Merge pull request #2411 from ujjwal96/ico-parser
ICO content view
2017-06-25 02:02:01 +02:00
Ujjwal Verma
3bccc2b8c6 Few ip formatting fixes 2017-06-25 04:09:45 +05:30
Ujjwal Verma
3977eecbb9 lint! 2017-06-25 01:13:09 +05:30
Ujjwal Verma
8ca29679df Added test 2017-06-25 01:05:20 +05:30
Maximilian Hils
bde64746a9 Merge pull request #2395 from MatthewShao/mitmweb-options
[WIP] Add RESTful API for mitmweb option
2017-06-24 19:54:04 +02:00
Matthew Shao
1863326220 Minor Update for /options API of mitmweb. 2017-06-24 10:18:00 +08:00
Ujjwal Verma
e81144e26b ICO Parser. Closes #2407 2017-06-24 03:28:44 +05:30
William Entriken
44d3188bf2 Slight wording change 2017-06-23 17:30:48 -04:00
Matthew Shao
c7ce7f84e6 Add test for POST /options API of mitmweb. 2017-06-23 10:58:59 +08:00
Matthew Shao
ab162361a1 Add POST /options RESTful API for mitmweb. 2017-06-23 10:44:09 +08:00
Ujjwal Verma
6380e74821 Removed redundant type 2017-06-22 06:28:11 +05:30
Ujjwal Verma
62403a5bd8 Fixes #2197 2017-06-19 03:18:38 +05:30
Ujjwal Verma
c5858463ca Fix ip formatting in websocket messages 2017-06-18 01:08:31 +05:30
Aldo Cortesi
e8939b8b9f console: keybindings tweaks
- consistent sort order
- preserve help on edit
2017-06-14 10:01:07 +12:00
Aldo Cortesi
2108bfb106 console: add console.key.edit.focus, bind to "enter" 2017-06-14 09:48:07 +12:00
Aldo Cortesi
e5f79abb4f console: add console.key.execute.focus, bind to x 2017-06-14 09:21:37 +12:00
Aldo Cortesi
254fe34d4c console: add console.key.unbind.focus, bind "d" key in keymap editor 2017-06-14 09:04:10 +12:00
Aldo Cortesi
788f0f5784 console: console.key.bind console.key.unbind commands 2017-06-14 08:34:34 +12:00
Matthew Shao
1177e6d907 Add typesepc_to_str function to mitmproxy/utils/typechck.py 2017-06-13 23:22:43 +08:00
Matthew Shao
2c0f6c2023 Minor Update and add test. 2017-06-13 23:03:07 +08:00
Aldo Cortesi
e6cf9ac9ab console: use master.shutdown rather than raising urwid.ExitMainLoop 2017-06-13 16:16:39 +12:00
Aldo Cortesi
79ce711784 console: extract console addons into consoleaddons.py 2017-06-13 16:15:22 +12:00
Aldo Cortesi
09128d9d1a console: keymap - list keys with modifiers separately 2017-06-13 16:10:40 +12:00
Aldo Cortesi
4a7cafee9e console: teach keymap to understand "space"
Urwid uses " ", which is not a great user experience.
2017-06-13 15:36:00 +12:00
Aldo Cortesi
ba49b55684 console: key binding viewer
Read-only for now.
2017-06-13 11:48:08 +12:00
Aldo Cortesi
88832f92a3 view: show a message if we remove more than one flow from view 2017-06-13 11:02:32 +12:00
Aldo Cortesi
56eb0441da commands: teach parser correct annotations for variable args
We should annotate with the base type, not the resulting sequence.
2017-06-13 10:26:03 +12:00
Aldo Cortesi
0fc24857e1 core: set command can now take multiple values
These are joined together with spaces before execution. This is a big
convenience boost on the command-prompt.
2017-06-13 09:47:50 +12:00
Aldo Cortesi
d95f28e6bf config: don't crash with a config file containing only comments 2017-06-13 09:43:12 +12:00
Aldo Cortesi
06cb68c799 Minor: formatting, make default color scheme solarized-dark 2017-06-13 09:30:33 +12:00
Matthew Shao
03bb0a09b9 Add tests for dump_dicts in optmanager.py. 2017-06-12 23:23:09 +08:00
Matthew Shao
aabc78350a Add GET /options RESTful API for mitmweb. 2017-06-12 20:26:46 +08:00
Oliver Graff
bd9d0bf6b1 Remove private key from PKCS12 file
Fixes #2349
2017-06-12 01:44:26 -07:00
Aldo Cortesi
a61178f28a console: console.eventlog.clear command, bind to "z" by default 2017-06-11 22:16:02 +12:00
Aldo Cortesi
9c57508fde console: extract "enter" to m_select
Regularise, kill dead code, remove keybindings now not needed.
2017-06-11 21:55:31 +12:00
Aldo Cortesi
60de6e6d1e console: delete stale help code 2017-06-11 21:22:55 +12:00
Aldo Cortesi
a482868cd6 console: extract "tab" to m_next pseudo-key
This makes tab work consistently across contexts, makes it globally bindable,
and gets rid of an un-needed command.
2017-06-11 21:20:12 +12:00
Aldo Cortesi
926b58ab4f console: Make global nav keys work within help 2017-06-11 18:57:02 +12:00
Aldo Cortesi
fdfd3ed937 console: Key binding help
Add a help field to key bindings, use that in contextual help display.
2017-06-11 18:19:23 +12:00
Aldo Cortesi
7caa7e7538 console: auto-generate context key binding help 2017-06-11 17:09:52 +12:00
Aldo Cortesi
816a51b6c8 console: help - remove hardcoded help everywhere 2017-06-11 13:28:31 +12:00
Aldo Cortesi
ecce260542 console: generalise grideditor help text, only show headers if we have them 2017-06-11 12:07:02 +12:00
Aldo Cortesi
a4833a5bbf console: grideditor - re-add column headers
Also remove special-case shortcut for user-agent. This is awkward in the new
structure. Down the track we can come up with a more regular mechanism like
snippets.
2017-06-11 11:57:44 +12:00
Aldo Cortesi
d7cc7e62a2 console: grideditor keybindings for r, R and e 2017-06-11 11:47:04 +12:00
Aldo Cortesi
f804495fd8 console: overlays shouldn't clear window titles 2017-06-11 11:17:19 +12:00
Aldo Cortesi
8427c2c4c7 console: extract grideditor keybindings
Also formalise the LayoutWidget interface a bit
2017-06-11 11:05:03 +12:00
Aldo Cortesi
2054739c1e console: grideditors don't display their own titles anymore 2017-06-10 10:09:48 +12:00
Aldo Cortesi
040238e255 console: layout pane heaaders
Controlled with the new console_layout_headers option
2017-06-10 09:54:03 +12:00
Thomas Kriechbaumer
02232b5149 Merge pull request #2379 from ujjwal96/kaitai
Update png.py. Stop reading after IEND
2017-06-06 00:51:03 +02:00
Ujjwal Verma
8b06dffafb Update png.py. Stop reading after IEND 2017-06-06 03:52:36 +05:30
Ujjwal Verma
dfa693ebd3 Fixes #2377 #2323 2017-06-06 03:16:00 +05:30
Thomas Kriechbaumer
0c91503b97 check tty after options parsing 2017-06-04 14:34:28 +02:00
Thomas Kriechbaumer
fd63386593 check for tty when running mitmproxy 2017-06-04 13:58:26 +02:00
Thomas Kriechbaumer
020e5a5c95 Merge pull request #2367 from mhils/issue-2329
Fix #2329
2017-06-02 14:18:56 +02:00
Ujjwal Verma
619566fc9f Fixes #2230 (#2361) 2017-06-02 13:31:12 +02:00
Maximilian Hils
5e17b5f62e minor improvements 2017-06-02 00:01:37 +02:00
Maximilian Hils
2ba0791810 fix #2329 2017-06-01 23:39:49 +02:00
David Shaw
7fef4efd81 sync with PyWBXMLDecoder master branch (#2366) 2017-06-01 20:37:18 +02:00
Ujjwal Verma
33f735ef50 Increase test coverage 2017-05-31 04:05:10 +05:30
Maximilian Hils
e7f7a608c6 Merge pull request #2358 from mhils/make-mypy-great-again
Fix mypy annotations, update mypy
2017-05-27 01:55:43 +02:00
Thomas Kriechbaumer
e5414ae2d3 http2: prevent KeyError 2017-05-26 17:59:19 +02:00
Thomas Kriechbaumer
4fb9cc1a69 fail-safe flowfilter 2017-05-26 17:58:38 +02:00
Maximilian Hils
eb5f37a7d1 fix mypy annotations 2017-05-26 16:14:20 +02:00
Thomas Kriechbaumer
4aeb6c6455 cleanup old tls_parser 2017-05-26 13:59:41 +02:00
Thomas Kriechbaumer
ee6ea31147 Merge pull request #2353 from ujjwal96/tls-parser
Using kaitai struct to parse tls client hello
2017-05-26 13:58:29 +02:00
Ujjwal Verma
9edfe1004b Added ksy file and updated make.sh 2017-05-26 06:38:32 +05:30
Ujjwal Verma
6637630e3e Using kaitai struct to parse tls client hello 2017-05-26 00:41:18 +05:30
Thomas Kriechbaumer
6214c6e185 fix openssl cipher_list bytes/str deprecation 2017-05-24 16:17:04 +02:00
Thomas Kriechbaumer
e77f07a36b cleanup 2017-05-24 14:23:36 +02:00
Thomas Kriechbaumer
ae7e9efb5c fix various fd/socket leaks 2017-05-24 14:09:41 +02:00
Thomas Kriechbaumer
928085c597 proxy tests: fix leaking sockets 2017-05-24 12:52:13 +02:00
Thomas Kriechbaumer
485ac4aa32 fix array.{fromstring,tostring} deprecation warning 2017-05-24 11:35:53 +02:00
Thomas Kriechbaumer
e82ed55516 close existing server_conn before replacing it 2017-05-24 11:35:53 +02:00
Thomas Kriechbaumer
ba05a29fbb TCP IPv6: fix leaking sockets 2017-05-24 11:35:35 +02:00
Maximilian Hils
2222f0b65a Merge pull request #2191 from LuD1161/Issue#2139
improve flow_detail help text
2017-05-22 12:06:07 +02:00
Thomas Kriechbaumer
79100593c2 Update options.py 2017-05-19 12:12:40 +08:00
Ujjwal Verma
5833b218b2 Increase net.tcp.py coverage (#2336) 2017-05-17 23:27:57 +08:00
Thomas Kriechbaumer
ca46b1e340 alpn_select_callback must be a callable 2017-05-17 10:38:31 +08:00
Thomas Kriechbaumer
3fadda2bba fix pragmas 2017-05-17 10:33:35 +08:00
Aldo Cortesi
a03a56f4dc net.tcp: server should be in shut-down state before start
The .configure and .tick events can be called before the server is running. If
an addon calls ctx.master.shutdown() at this point, the server hangs forever
waiting on __is_shutdown_set. This patch fixes this by starting off with it
already set.
2017-05-14 10:28:12 +12:00
Ujjwal Verma
dc40fc4933 Changes in setup.cfg 2017-05-12 21:09:50 +05:30
Ujjwal Verma
d47de94303 Increase io.compat coverage 2017-05-12 21:09:50 +05:30
Ujjwal Verma
9d0771bae5 net.http.url.py changes 2017-05-12 21:09:50 +05:30
charlesdhdt
9f8e83259e Fixed LDAP Auth (#2333) 2017-05-12 15:37:00 +02:00
Maximilian Hils
8414df1e1a fix type annotation 2017-05-11 10:15:28 +02:00
Ujjwal Verma
3e85dfd242 Handle head request on onboarding app: Fix #2324 (#2325) 2017-05-09 20:08:42 +02:00
Ritiek Malhotra
4c2404049c Change to only Apps & VPN 2017-05-08 06:43:44 +05:30
Ritiek Malhotra
e07d2b0660 Add instructions for Windows and Android 2017-05-07 08:13:45 +05:30
Ritiek Malhotra
98fd704f67 Add apple info from docs 2017-05-07 07:44:58 +05:30
Ritiek Malhotra
39356e8f34 Add dynamic apple instructions 2017-05-07 02:07:41 +05:30
Maximilian Hils
d734f6bbd6 Merge pull request #2286 from charlesdhdt/master
Added LDAP Auth
2017-05-04 16:24:57 +02:00
Charles d'Hondt
f67d9adc31 Added ldap mock test 2017-05-04 13:25:15 +02:00
Maximilian Hils
315daa042a fix #2310 2017-05-03 17:38:17 +02:00
Aldo Cortesi
2659b52209 console: add a two-pane layout
- Replace options.console_eventlog with options.console_layout
- This can be "single", "vertical" and "horizontal"
- At the base of the primary pane is the flowlist. At the base of the secondary
pane is the event log.
- Any of the other primary windows can be opened in each of the panes.

For now, I've bound "-" to the flow layout switch, "shift tab" to the layout
pane switch, and "P" to open the currently focused flow in whichever pane
you're in. These are just temporary  - we'll reassess the default bindings
carefully once the keybindings work is complete.
2017-05-03 14:55:02 +12:00
Aldo Cortesi
0f4d94b31c console: Make EventLog its own control, bind to "E" 2017-05-02 22:42:04 +12:00
Charles d'Hondt
a68808294d lint checks 2017-05-02 10:35:56 +02:00
Aldo Cortesi
dcae79e017 console: various bugs
- Action bar now correctly gets the palette background
- Viewing connection details for a killed flow no longer crashes
- Editing options with selections no longer crashes
2017-05-02 12:56:27 +12:00
Aldo Cortesi
4e39d387d3 commands: options control
Use the new commands to bind the Options view in console.
2017-05-02 12:18:24 +12:00
Aldo Cortesi
9e58c36639 console: globally respected nav commands
Use this to bind nav keys with commands throughout.
2017-05-02 12:18:24 +12:00
Aldo Cortesi
2f52d8b9ad commands: view.setval, view.getval, view.setval.toggle
Use these to bind the last of the key for flowview. Flow views are now 100%
keybound.
2017-05-01 19:52:24 +12:00
Aldo Cortesi
7d56ec9d49 commands: flow.encode, flow.decode, flow.encode.toggle
Use this to bind "z" in flowview.
2017-05-01 18:31:22 +12:00
Aldo Cortesi
542a998174 commands: console.bodyview, console.choose
Use this to replace the "v" for view shortcut key in flowview.
2017-05-01 17:56:21 +12:00
Aldo Cortesi
1ea4a5a48e keymap: keys can now bind to multiple contexts
Use this to map the majority of the keys in flowview.
2017-05-01 16:52:58 +12:00
Aldo Cortesi
670d1e408b command: flow.set
Use this to replace the flow edit components of flowview entirely.
2017-05-01 14:56:57 +12:00
Aldo Cortesi
46373977e2 Refactor console
- All top-level window objects are now persistent. It turns out that urwid
keeps a reference to these even after they're no longer used, so they were
leaking before.
- Revamp editors to work with console commands, and start reworking bindings
for flowview.
2017-05-01 14:12:22 +12:00
Maximilian Hils
fccc9bc45e Merge pull request #2271 from mhils/mitmweb-connection-indicator
[web] add connection indicator [WIP]
2017-04-30 15:23:20 +02:00
Aldo Cortesi
685487d33c commands: view.order.options, view.marked.toggle, view.create
And use these commands to remove the last hard-coded keybindings from flow
list. This means the flow list is now 100% command-driven, which is very
exciting.
2017-04-30 22:45:31 +12:00
Aldo Cortesi
a570caccbd commands: view.load
Plus replace the flow list keybinding.
2017-04-30 22:02:29 +12:00
Aldo Cortesi
3cd93567f5 commands: support *args for commands
Use this to simplify meta-commands in console, and to create a console_choose
command that prompts the user for a choice, and then executes a command with
variable substitution.
2017-04-30 21:24:00 +12:00
Aldo Cortesi
bcbe87bb09 Move export to addon, kill Python and Locust export
Also add a "raw" export format.

The Python and Locust exports are hard to maintain, their tests are extremely
brittle, they didn't have full test coverage, and are by my guess very rarely
used. I feel the Locust export should certainly be an externally maintained
addon. The Python/requests export can come back if someone cares enough, and it
can be structured in a way we can maintain.
2017-04-30 17:53:30 +12:00
Aldo Cortesi
befbe88d96 commands: cut.clip copies cuts to system clipboard 2017-04-30 14:42:37 +12:00
Aldo Cortesi
ed62bbad1d Add help text to --commands output 2017-04-30 14:20:21 +12:00
Aldo Cortesi
075d452a6d cut: more flexible cut specification based on attribute paths
Also support certificate types, which are converted to ASCII-encoded PEM format.
2017-04-30 14:05:45 +12:00
Aldo Cortesi
7ffb2c7981 cut: use csv module to encode multi-values for saving
Also add q.text, q.raw_content, s.text, s.raw_content selectors
2017-04-30 11:48:32 +12:00
Maximilian Hils
97a00728a8 [web] add connection tests 2017-04-29 19:43:59 +02:00
Maximilian Hils
8f1b763082 [web] add connection indicator 2017-04-29 19:34:51 +02:00
Aldo Cortesi
4b568f99d6 Introduce cuts: a flow dissector
This PR introduces the cuts addon, a flow dissector that  allows you to select
and operate on specific components of flows. It also adds the first consumer
for cuts - the cuts.save command.

Save the content of the focus to /tmp/foo:

    cuts.save s.content|@focus /tmp/foo

Save the URL and response content-type headers for all flows currently shown to
file, comma-separated, one flow per line:

    cuts.save s.url,q.header[content-type]|@focus /tmp/foo

We also use this to replace the body save shortcut in the console flowlist.
2017-04-29 23:32:31 +12:00
Aldo Cortesi
d439b34511 command: script.run
Plus the flowlist binding adjustments.
2017-04-29 13:24:19 +12:00
Aldo Cortesi
1d5eedcc9c command: flow.revert
Plus matching binding in flowlist.
2017-04-29 11:48:25 +12:00
Aldo Cortesi
f908ea220f commands: "replay.client.file" and "replay.server.file" 2017-04-29 11:34:50 +12:00
Aldo Cortesi
a92017a6c1 Rework client and server replay
- Add client.replay [flows], client.replay.stop
- Add server.replay [flows], server.replay.stop
- The corresponding options for file loading are only read on startup, further
changes are ignored. In interactive contexts, replay is started with the
commands, not through option changes.
- Deprecate flow.replay, use replay.client instead
2017-04-29 11:02:36 +12:00
Aldo Cortesi
7317ea134e command: flow.kill, flow.replay
Plus the matching bindings in the flow list.
2017-04-29 09:58:32 +12:00
Aldo Cortesi
f21a970f29 commands: marking
Add "view.mark [flows] bool" and "view.mark.toggle [flows]". Use this to rebind
marking keys in flowlist.
2017-04-29 09:14:44 +12:00