Commit Graph

1906 Commits

Author SHA1 Message Date
Aldo Cortesi
e59ba13417 Use deferral mechanism for cfg file options
Fixes #3162
2018-05-27 10:12:24 +12:00
Aldo Cortesi
b6e1c4bb9d cibuild: tweak docker tag, only upload prod builds to pypi 2018-05-25 12:46:56 +12:00
Aldo Cortesi
ff92962c51 cibuild: start building version awareness 2018-05-25 10:50:48 +12:00
Aldo Cortesi
fdf0d9974e cibuild: more accurate PR test, better dump output 2018-05-24 22:58:33 +12:00
Aldo Cortesi
e9b19dba8a cibuild: fix docker upload condition, expand tests 2018-05-24 22:32:31 +12:00
Aldo Cortesi
c91b511bc7 cibuild: extract upload conditions 2018-05-24 22:18:01 +12:00
Aldo Cortesi
0afff3a952 cibuild: add docker tag calculation to environ 2018-05-24 21:59:45 +12:00
Aldo Cortesi
bae4a3393f release: include release directory in coverage
With lots of exclusions for now
2018-05-24 19:03:15 +12:00
Aldo Cortesi
8fceaca6b8 cibuild: Consolidate build environment and add tests 2018-05-24 11:06:43 +12:00
Aldo Cortesi
064929b559 clientreplay: fix racy tests harder 2018-05-18 09:51:56 +12:00
Thomas Kriechbaumer
976ab0c466 websocket: inject messages via flow 2018-05-16 21:50:15 +02:00
Thomas Kriechbaumer
9979be6487 fix py36/37 escaping issues
closes #3122

https://bugs.python.org/issue16285
https://bugs.python.org/issue12910
21024f0662
2018-05-14 10:41:46 +02:00
Aldo Cortesi
58ff51da10 command: handle string without terminal escaped char
Fixes #2810
2018-05-12 14:02:20 +12:00
Aldo Cortesi
88fe26997c script: revamp ephemeral script running
Fixes 2838
2018-05-12 12:35:24 +12:00
Aldo Cortesi
482043cdcf commands: handle type errors on startup
fixes #3088
fixes #2787
2018-05-12 11:10:14 +12:00
Aldo Cortesi
6dff8c58ad commands: if no explicit return type is specified, assume None
This is going to be a super common error for addon authors, so we might as well
handle it.
2018-05-12 10:15:08 +12:00
Aldo Cortesi
8c63a8818d keymap: read keys from CONFDIR/keys.yaml by default 2018-05-10 16:06:52 +12:00
Aldo Cortesi
9830e5b597 cadir -> confdir
We store a lot more than just the CAs in our configuration directory. Clarify
the option name.
2018-05-10 11:40:33 +12:00
Aldo Cortesi
3438912236 console keybindings: define YAML-based format for console key binding persistence 2018-05-10 11:30:51 +12:00
Aldo Cortesi
7d9b626d2e clientplayback: try to fix flaky replay test 2018-05-08 16:33:41 +12:00
Aldo Cortesi
af1a4ffdcd Ditch the addon stdout wrapper
This results in a 30% improvement in our core request throughput.

Fixes #3102
2018-05-08 14:26:41 +12:00
Aldo Cortesi
717fbaa990 optmanager: refactor for coverage and clarity 2018-05-08 14:24:41 +12:00
Aldo Cortesi
1b0eed19d1 benchmarks: quick-run scripts for mitmproxy and mitmdump 2018-05-08 11:08:37 +12:00
Aldo Cortesi
9b51393b6d benchmark: keep track of requests/responses seen by proxy 2018-05-08 11:00:20 +12:00
Aldo Cortesi
f7d7e31f06 options: add the concept of deferred settings
We've had a perpetual sequencing problem with addon startup. Users need to be
able to specify options to addons on the command-line, before addons are
actually loaded. This is only exacerbated with the new async core, where load
order can't be relied on.

This patch introduces deferred options. Options passed with "--set" on the
command line are deferred if they are unknown, and are automatically applied by
the addon manager once matching addons are registered and their options are defined.
2018-05-08 10:56:00 +12:00
Maximilian Hils
752a367436 remove leftover processing_complete 2018-05-06 15:47:12 +02:00
Aldo Cortesi
60acbd79b9 Remove allowremote addon, add an improved take called block
We now have two options: block_global blocks global networks, block_private
blocks private networks. The block_global option is true by default, and
block_private is false by default. The addon name is "block" so the options are
correctly prefixed.

Also make option documentation precise, reduce verbosity of logs.
2018-05-06 12:43:25 +12:00
Aldo Cortesi
c53bc39c95
Merge pull request #3099 from Kriechi/fix-3024
fix #3024
2018-05-06 11:27:14 +12:00
Thomas Kriechbaumer
54e2daa21e fix #3024 2018-05-05 14:38:02 +02:00
Aldo Cortesi
6bd2d5059b Remove custom events
These were always weird, and were added to support exit when processing
completed to mitmdump. We now have better ways to do this.
2018-05-03 09:42:51 +12:00
Aldo Cortesi
f380a77dee Remove the tick event
Mitmproxy: the tickless wonder.
2018-05-02 14:29:15 +12:00
Aldo Cortesi
38ff8109fb taddons: remove has_event
We no longer use this anywhere, so ditch it.
2018-05-02 11:31:28 +12:00
Aldo Cortesi
22a4b1d5d4 Redesign keepserving
- Instead of listening for a pseudo-event, we periodically check whether client
replay, server replay or file reading is active.
- Adjust server replay not to
use tick.
- Adjust readfile to expose a command to check whether reading is in progress.
2018-05-02 11:26:40 +12:00
Aldo Cortesi
00d790fe84 commands: clarify command call interface, fix web app replay 2018-05-02 08:36:15 +12:00
Aldo Cortesi
6d27b28b85 client replay: expad and consolidate tests 2018-05-01 08:47:26 +12:00
Aldo Cortesi
236a2fb6fd client replay: re-design
Re-design the way client replay works. Before, we would fire up a thread,
replay, wait for the thread to complete, get the next flow, and repeat the
procedure. Now, we have one replay thread that starts when the addon starts,
which pops flows off a thread-safe queue. This is much cleaner, removes the
need for busy tick, and sets the scene for optimisations like server connection
reuse down the track.
2018-04-30 17:17:03 +12:00
Aldo Cortesi
28d53d5a24 client replay: move all client replay-related code into addon 2018-04-27 16:34:56 +12:00
Aldo Cortesi
a4a48a96d6
Merge pull request #3078 from kajojify/expanduser
Fix #3002. Auto-expanding for tilda.
2018-04-26 21:02:10 +12:00
Aldo Cortesi
b18e0fa2bc
Merge pull request #3071 from cortesi/tickless
asyncio: shift script reloading out of the tick event
2018-04-26 11:47:07 +12:00
Miroslav
ea6fab09b5 Fix #3002. Auto-expanding 2018-04-26 01:35:44 +03:00
Maximilian Hils
19ceb6f49a fix pytest invocation errors 2018-04-25 14:00:50 +02:00
Aldo Cortesi
95e690ba31 test: shift test_data out of our public API 2018-04-23 19:16:33 +12:00
Aldo Cortesi
0ba10b6109 addons/script: improve relability of reload test
The granularity of mtime is surprisingly bad. Make the tests more robust
against this, and promote has_log back to a public method, now that we have a
few legitimate examples.
2018-04-23 13:19:20 +12:00
Aldo Cortesi
44016a0de5 asyncio: shift script reloading out of the tick event
The tick event is a nasty compromise, left over from when we didn't have an
event loop. This is the first patch in a series that explores moving our
built-in addons to managing coroutines on the eventloop directly for periodic
tasks.
2018-04-23 10:28:18 +12:00
Aldo Cortesi
ef4db52887 Ignore signal errors on windows + various fixes
- Ignore the NotImplementedError raised by add_signal_handler on Windows.
- Entrypoints return an integer exit code, or None. Adjust our type annotations
and code to suit.

Fixes #3061
2018-04-17 08:20:52 +12:00
Aldo Cortesi
565146311a asyncio: clarify shutdown semantics
This patch clarifies proxy shutdown, and specifies that the master.shutdown()
method is thread-save.
2018-04-16 10:16:51 +12:00
Aldo Cortesi
850c855495 Clean up test/helper_tools directory 2018-04-16 09:20:22 +12:00
Aldo Cortesi
b663a224a3 Improve benchmarking
- The benchmark addon now manages setting up and tearing down the backend and
traffic processes itself.
- Use wrk instead of hey. I get more consistent results with this tool, and hey
shows a strange tail-latency bump that seems artificial.
- Make termination behaviour simpler. The bencmark revealed a bug where .done
events were not called if the proxy was shut down by an addon.
2018-04-16 09:19:14 +12:00
Aldo Cortesi
5f74adc2df
Merge pull request #3059 from obscure76/enh/issue-3053
fix Python 3.6 variable type annotations #3053
2018-04-16 07:35:20 +12:00
oscure76
0e984e1442 fix Python 3.6 variable type annotations #3053 2018-04-14 16:24:41 -07:00