Commit Graph

1614 Commits

Author SHA1 Message Date
Thomas Kriechbaumer
2faaa0b2a2 connections tests: fix leaking sockets 2017-05-24 13:21:58 +02:00
Thomas Kriechbaumer
928085c597 proxy tests: fix leaking sockets 2017-05-24 12:52:13 +02:00
Thomas Kriechbaumer
c9529ffe45 http1 tests: fix leaking sockets 2017-05-24 11:43:57 +02:00
Thomas Kriechbaumer
618044c637 http2 tests: fix leaking sockets 2017-05-24 11:43:50 +02:00
Thomas Kriechbaumer
c4f028d701 websocket tests: fix leaking sockets 2017-05-24 11:35:53 +02:00
Thomas Kriechbaumer
0bd3d0ff4c upgrade pytest and remove our pytest.raises wrapper 2017-05-24 11:35:53 +02:00
Thomas Kriechbaumer
4961d782ed fix typo 2017-05-24 11:35:53 +02:00
Thomas Kriechbaumer
b4f65049f2 auto-close file after reading 2017-05-24 11:35:53 +02:00
Maximilian Hils
5626c384ab Merge pull request #2342 from MatthewShao/jest-dev
[web] Add  _flow.js for component tests.
2017-05-22 10:00:35 +02:00
Matthew Shao
2935ff5f3a [web] Set some value as constant, so _tflow.js would not change. 2017-05-22 08:53:59 +08:00
Matthew Shao
2c38fddea5 [web] TFlow minor improved. 2017-05-21 20:59:19 +08:00
Matthew Shao
369e1f1eee [web] Fix minior lint problem. 2017-05-19 09:56:52 +08:00
Matthew Shao
50c0703424 [web] Add _tflow.js generator in test_app.py 2017-05-19 09:45:30 +08:00
Ujjwal Verma
5833b218b2 Increase net.tcp.py coverage (#2336) 2017-05-17 23:27:57 +08:00
Ujjwal Verma
37053f0efc Increased mitmproxy.controller coverage 2017-05-17 11:06:40 +05:30
Ujjwal Verma
204d06f93f Increased xml_html contentview coverage 2017-05-17 11:05:49 +05:30
Ujjwal Verma
8eea05fcaf Increase contentviews api coverage 2017-05-17 11:04:43 +05:30
Ujjwal Verma
d47de94303 Increase io.compat coverage 2017-05-12 21:09:50 +05:30
Ujjwal Verma
766c5caea8 Increased net.http.cookies.py and net.http.message.py coverage 2017-05-12 21:09:50 +05:30
charlesdhdt
9f8e83259e Fixed LDAP Auth (#2333) 2017-05-12 15:37:00 +02:00
Ujjwal Verma
3e85dfd242 Handle head request on onboarding app: Fix #2324 (#2325) 2017-05-09 20:08:42 +02:00
Maximilian Hils
d734f6bbd6 Merge pull request #2286 from charlesdhdt/master
Added LDAP Auth
2017-05-04 16:24:57 +02:00
Charles d'Hondt
154e8ac0fc fixed lint 2017-05-04 13:39:48 +02:00
Charles d'Hondt
f67d9adc31 Added ldap mock test 2017-05-04 13:25:15 +02:00
Maximilian Hils
315daa042a fix #2310 2017-05-03 17:38:17 +02:00
Aldo Cortesi
2659b52209 console: add a two-pane layout
- Replace options.console_eventlog with options.console_layout
- This can be "single", "vertical" and "horizontal"
- At the base of the primary pane is the flowlist. At the base of the secondary
pane is the event log.
- Any of the other primary windows can be opened in each of the panes.

For now, I've bound "-" to the flow layout switch, "shift tab" to the layout
pane switch, and "P" to open the currently focused flow in whichever pane
you're in. These are just temporary  - we'll reassess the default bindings
carefully once the keybindings work is complete.
2017-05-03 14:55:02 +12:00
Aldo Cortesi
dcae79e017 console: various bugs
- Action bar now correctly gets the palette background
- Viewing connection details for a killed flow no longer crashes
- Editing options with selections no longer crashes
2017-05-02 12:56:27 +12:00
Aldo Cortesi
4e39d387d3 commands: options control
Use the new commands to bind the Options view in console.
2017-05-02 12:18:24 +12:00
Aldo Cortesi
9e58c36639 console: globally respected nav commands
Use this to bind nav keys with commands throughout.
2017-05-02 12:18:24 +12:00
Aldo Cortesi
2f52d8b9ad commands: view.setval, view.getval, view.setval.toggle
Use these to bind the last of the key for flowview. Flow views are now 100%
keybound.
2017-05-01 19:52:24 +12:00
Aldo Cortesi
7d56ec9d49 commands: flow.encode, flow.decode, flow.encode.toggle
Use this to bind "z" in flowview.
2017-05-01 18:31:22 +12:00
Aldo Cortesi
1ea4a5a48e keymap: keys can now bind to multiple contexts
Use this to map the majority of the keys in flowview.
2017-05-01 16:52:58 +12:00
Aldo Cortesi
670d1e408b command: flow.set
Use this to replace the flow edit components of flowview entirely.
2017-05-01 14:56:57 +12:00
Aldo Cortesi
685487d33c commands: view.order.options, view.marked.toggle, view.create
And use these commands to remove the last hard-coded keybindings from flow
list. This means the flow list is now 100% command-driven, which is very
exciting.
2017-04-30 22:45:31 +12:00
Aldo Cortesi
a570caccbd commands: view.load
Plus replace the flow list keybinding.
2017-04-30 22:02:29 +12:00
Aldo Cortesi
3cd93567f5 commands: support *args for commands
Use this to simplify meta-commands in console, and to create a console_choose
command that prompts the user for a choice, and then executes a command with
variable substitution.
2017-04-30 21:24:00 +12:00
Aldo Cortesi
bcbe87bb09 Move export to addon, kill Python and Locust export
Also add a "raw" export format.

The Python and Locust exports are hard to maintain, their tests are extremely
brittle, they didn't have full test coverage, and are by my guess very rarely
used. I feel the Locust export should certainly be an externally maintained
addon. The Python/requests export can come back if someone cares enough, and it
can be structured in a way we can maintain.
2017-04-30 17:53:30 +12:00
Aldo Cortesi
befbe88d96 commands: cut.clip copies cuts to system clipboard 2017-04-30 14:42:37 +12:00
Aldo Cortesi
ed62bbad1d Add help text to --commands output 2017-04-30 14:20:21 +12:00
Aldo Cortesi
075d452a6d cut: more flexible cut specification based on attribute paths
Also support certificate types, which are converted to ASCII-encoded PEM format.
2017-04-30 14:05:45 +12:00
Aldo Cortesi
7ffb2c7981 cut: use csv module to encode multi-values for saving
Also add q.text, q.raw_content, s.text, s.raw_content selectors
2017-04-30 11:48:32 +12:00
Aldo Cortesi
4b568f99d6 Introduce cuts: a flow dissector
This PR introduces the cuts addon, a flow dissector that  allows you to select
and operate on specific components of flows. It also adds the first consumer
for cuts - the cuts.save command.

Save the content of the focus to /tmp/foo:

    cuts.save s.content|@focus /tmp/foo

Save the URL and response content-type headers for all flows currently shown to
file, comma-separated, one flow per line:

    cuts.save s.url,q.header[content-type]|@focus /tmp/foo

We also use this to replace the body save shortcut in the console flowlist.
2017-04-29 23:32:31 +12:00
Aldo Cortesi
d439b34511 command: script.run
Plus the flowlist binding adjustments.
2017-04-29 13:24:19 +12:00
Aldo Cortesi
1d5eedcc9c command: flow.revert
Plus matching binding in flowlist.
2017-04-29 11:48:25 +12:00
Aldo Cortesi
f908ea220f commands: "replay.client.file" and "replay.server.file" 2017-04-29 11:34:50 +12:00
Aldo Cortesi
a92017a6c1 Rework client and server replay
- Add client.replay [flows], client.replay.stop
- Add server.replay [flows], server.replay.stop
- The corresponding options for file loading are only read on startup, further
changes are ignored. In interactive contexts, replay is started with the
commands, not through option changes.
- Deprecate flow.replay, use replay.client instead
2017-04-29 11:02:36 +12:00
Aldo Cortesi
7317ea134e command: flow.kill, flow.replay
Plus the matching bindings in the flow list.
2017-04-29 09:58:32 +12:00
Aldo Cortesi
f21a970f29 commands: marking
Add "view.mark [flows] bool" and "view.mark.toggle [flows]". Use this to rebind
marking keys in flowlist.
2017-04-29 09:14:44 +12:00
Aldo Cortesi
217addbf31 commands: view.go
bind G to "view.go -1"
bind g to "view.go 0"
2017-04-29 08:40:38 +12:00
Aldo Cortesi
0b090f7ae1 Commands, core update event
This patch:

- Introduces a core update() event that should be invoked whenever flows are
changed outside of the normal lifecycle.
- Extend view.resolve to know about @all, which matches all flows in the view.
- Add a core flow.resume comand, which resumes flows and broadcasts an update event.
- Define flow list bindings for:

A -> flow.resume @all
a -> flow.resume @focus
d -> view.remove @focus
z -> view.remove @all
2017-04-29 08:01:00 +12:00
Charles d'Hondt
65202f5f1c Added ldapsss vef 2017-04-28 17:00:21 +02:00
Charles d'Hondt
6d3837fd54 fix 2017-04-28 16:44:50 +02:00
Charles d'Hondt
29c1f303d6 Fixed typo 2017-04-28 16:23:32 +02:00
Charles d'Hondt
5fb18ad275 Added LDAP Auth 2017-04-28 16:19:27 +02:00
Aldo Cortesi
6af1a49464 commands: add a command.command decorator
Use this for our built-ins and the console commands.
2017-04-28 15:07:52 +12:00
Aldo Cortesi
7ff84673fd Support unary flow command arguments
Use this to add a console.view.flow command and bind "enter" in flowlist.
2017-04-28 14:05:35 +12:00
Aldo Cortesi
cfae95f5c3 console: start migrating hotkeys to keymap
This shifts a set of flow list keys to keymaps. Much more to come.
2017-04-28 12:10:48 +12:00
Aldo Cortesi
ce01cb9c09 command: partials
Support partial commands in console.command, use it to keybind "i" for
intercept, which prompts the user with ":set intercept=".
2017-04-28 11:24:28 +12:00
Aldo Cortesi
8a07059cf4 commands: add the core command addon, and the command "set"
The set command sets an option using the same syntax as commandline --set.
2017-04-28 10:41:44 +12:00
Aldo Cortesi
be1b76b975 console: add a keymap
This sketches out a keymap system for consone, and adds the first few top-level
commands and mappings.
2017-04-28 10:21:15 +12:00
Aldo Cortesi
18edc11145 console: interactive command browser
Simple browser that lets users view and select commands interactively.
Key binding for this is still to be sorted out.
2017-04-28 07:43:58 +12:00
Aldo Cortesi
fde1159ae3 Test coverage ++ 2017-04-27 18:43:44 +12:00
Aldo Cortesi
97000aa85c command: save.file flowspec path -> None
Our first user-facing command. The following commands do the obvious things:

save.file @marked /tmp/flows
save.file @focus /tmp/flows
save.file @hidden /tmp/flows
save.file "~m get" /tmp/flows
2017-04-27 17:05:00 +12:00
Aldo Cortesi
b7afcb5dc2 addons.streamfile -> addons.save
Options:

    streamfile -> save_stream_file
    streamfile_filter -> save_stream_filter
2017-04-27 15:58:54 +12:00
Aldo Cortesi
8c4810f606 console: flow resolution command
This is our first built-in command, which will be used by very many other
commands.

Also add a --commands option to dump all commands, analogous to --options.
2017-04-27 15:27:51 +12:00
Aldo Cortesi
169baabcab Basic outline of the command subsystem
- Add a command manager
- Sketch out the type system with a few simple supported types
2017-04-27 11:09:40 +12:00
Aldo Cortesi
2a46f3851a Merge pull request #2265 from cortesi/addons
Addons and addon testing
2017-04-27 07:40:14 +12:00
Maximilian Hils
0f9081b18d Merge pull request #2269 from mhils/issue-2250
fix #2250, add type info to cookie module
2017-04-26 14:42:48 +02:00
Maximilian Hils
87610cc8b2 fix #2250, add type info to cookie module 2017-04-26 14:17:14 +02:00
Maximilian Hils
d5ea08db62 Merge pull request #2258 from mhils/readfile
Integrate readstdin into readfile
2017-04-26 13:40:51 +02:00
Maximilian Hils
0a8e54edea Merge pull request #2268 from mhils/issue-2257
fix #2257
2017-04-26 13:39:19 +02:00
Maximilian Hils
1aa6d9d324 fix #2257 2017-04-26 12:26:49 +02:00
Maximilian Hils
ca2827886a separate reading from stdin into its own addon 2017-04-26 12:11:37 +02:00
Maximilian Hils
df7701bb6d fix #2228 2017-04-26 11:32:24 +02:00
Maximilian Hils
b3a1143338 integrate readstdin into readfile addon 2017-04-26 10:56:32 +02:00
Maximilian Hils
2b500f234f typecheck: add support for typing.Any 2017-04-26 10:54:36 +02:00
Aldo Cortesi
5327756377 Addons and addon testing
- Fix some loading sequence bugs affecting command-line script invocation
- Allow addons to over-ride existing options (with a warning). We need this for
reloading.
- Convert har_dump to new-style arguments, fix and re-instate its test suite.
- Covnert miscelaneous other exmples to new-style args.
2017-04-26 19:56:33 +12:00
Aldo Cortesi
e32efcae49 Merge pull request #2262 from Kriechi/reduced-coverage++
improve tests (extracted from #2011)
2017-04-26 19:52:33 +12:00
Aldo Cortesi
b72f139093 configure(options, updated) -> configure(updated)
Options are now available globally on ctx, so the first argument of configure
is redundant.
2017-04-26 11:01:27 +12:00
Aldo Cortesi
f90b4c2ff0 Move options into ctx
Many addons currently save options on configure(), either as individual options
or sometimes by saving the entire options object. The current options should
simply be available on the ctx object, simplifying state management for addons
considerably.
2017-04-26 10:25:56 +12:00
Aldo Cortesi
7aa2081894 Remove watchdog, solidify script testing
- Remove the watchdog dependency. We now just stat the script file every 2
seconds to check for an updated mtime.
- Further solidify our script testing, and in particular make the example tests
nicer. These should exemplify how we want users to test their own addon
scripts. More work on addon testing to follow.
2017-04-26 09:01:40 +12:00
Thomas Kriechbaumer
a35a377cbb improve tests 2017-04-25 20:49:49 +02:00
Aldo Cortesi
e6eeab6094 Revamp how addons work
- Addons now nest, which means that addons can manage addons. This has a number
of salutary effects - the scripts addon no longer has to poke into the global
addons list, we no longer have to replace/remove/boot-outof parent addons when
we load scripts, and this paves the way for making our top-level tools into
addons themselves.
- All addon calls are now wrapped in a safe execution environment where
exceptions are caught, and output to stdout/stderr are intercepted and turned
into logs.
- We no longer support script arguments in sys.argv - creating an option
properly is the only way to pass arguments. This means that all scripts are
always directly controllable from interctive tooling, and that arguments are
type-checked.

For now, I've disabled testing of the har dump example - it needs to be moved
to the new argument handling, and become a class addon. I'll address that in a
separate patch.
2017-04-25 22:13:44 +12:00
Ujjwal Verma
742127ef7b Fixes #2109 2017-04-09 18:55:20 +05:30
Robert C Jensen
7365f18542 fixes ipv6 authority form parsing in CONNECT 2017-04-07 15:46:21 -04:00
Nikhil Soni
442999c75c Add tests for set_focus 2017-03-29 18:34:17 +05:30
Aldo Cortesi
65f0885bd6 addon loader: add boot_into, which replaces returning from start()
While we're here, expand test coverage for addonmanager to 100%, and promote to
individual coverage.
2017-03-25 10:48:12 +13:00
Aldo Cortesi
541c1e8b9f addons: start -> load throughout 2017-03-25 10:48:12 +13:00
Thomas Kriechbaumer
ef9f0e22ea bump h2 2017-03-24 21:27:14 +01:00
Aldo Cortesi
335861f490 Merge pull request #2207 from nikofil/scripts-redirect-stdout
scripts: redirect stdout to ctx.log.warn
2017-03-24 14:58:33 +13:00
Nikos Filippakis
abf291b0f9 scripts: redirect stdout to ctx.log.warn
Redirect messages written to stdout in scripts to ctx.log.warn.
(closes #1530)

Signed-off-by: Nikos Filippakis <nikolaos.filippakis@cern.ch>
2017-03-24 02:35:02 +01:00
Aldo Cortesi
0d3e2c6adf optmanager: make serialization and loading more robust
Also handle errors in console options manager.
2017-03-24 09:37:17 +13:00
Thomas Kriechbaumer
907825714b move examples tests (#2199) 2017-03-22 12:02:18 +01:00
Aldo Cortesi
4ca78604af Factor out an io module
Include tnetstring - we've made enough changes that this no longer belongs in
contrib.
2017-03-20 12:50:09 +13:00
Aldo Cortesi
04f1a4443d options: reraise when there's an error with a partial update
Fixes #2174
2017-03-19 14:36:23 +13:00
Aldo Cortesi
43e7b9d68c options: choices for content views and tighten console order spec
Also clean up some leftovers from filter option name change.
2017-03-19 13:39:28 +13:00
Aldo Cortesi
22d7c31ea8 dumper, mitmdump: kill filtstr
Dumper now also uses view_filter. Passing arguments to mitmdump is now
precisely equivalent to setting both filestream_filter and view_filter.
2017-03-19 13:21:46 +13:00
Aldo Cortesi
475a4e3eb0 streamfile: add streamfile_filter and use it instead of filtstr 2017-03-19 13:08:26 +13:00
Aldo Cortesi
21794c7bbe optmanager: test coverage 2017-03-19 11:29:41 +13:00
Aldo Cortesi
49b0a67eb9 console options: add an overlay grid editor for sequence options 2017-03-19 10:32:22 +13:00