Commit Graph

8123 Commits

Author SHA1 Message Date
Aldo Cortesi
44016a0de5 asyncio: shift script reloading out of the tick event
The tick event is a nasty compromise, left over from when we didn't have an
event loop. This is the first patch in a series that explores moving our
built-in addons to managing coroutines on the eventloop directly for periodic
tasks.
2018-04-23 10:28:18 +12:00
Thomas Kriechbaumer
f6b606b364
Merge pull request #3070 from CorTal/Fix#3066
Fix asyncio.call_soon -> asyncio.ensure_future
2018-04-18 16:38:52 +02:00
CorTal
ab01406585 call_soon->ensure_future 2018-04-18 10:31:43 +02:00
Aldo Cortesi
6d5c8781e2
Merge pull request #3069 from cortesi/mweb
A grab-bag of mitmweb-related fixes
2018-04-18 09:17:14 +12:00
Aldo Cortesi
e6538cdb1f mitmweb: don't blort all over my config file
Temporarily disable config saving until we can look at this more closely.
Config saving should:

- Only happen when explicitly requested by the user
- Only save changed values
2018-04-18 09:01:42 +12:00
Aldo Cortesi
bca0275427 tcp: As of Python 3.5, EINTR is retried automatically by select
See https://www.python.org/dev/peps/pep-0475/
2018-04-18 08:22:26 +12:00
Aldo Cortesi
8396244729 mitmweb: shift to use new run_loop wrapper on master
Fixes #3067
2018-04-18 08:21:08 +12:00
Aldo Cortesi
633023701b
Merge pull request #3064 from cortesi/mexit
asyncio: fix exit behaviour for console
2018-04-18 07:58:51 +12:00
Aldo Cortesi
df83c0eef7 asyncio: fix exit behavior for console
- Add a master.run_loop function. This encapsulates our run behaviour so that
it can be used by implementations that need to manage their own run loop (like urwid).
- Shift crash exit message to the common core. I'm not convinced we really need
this, but if we want it it should be centralised.
- Clean up an extra exception that can be thrown by asyncio itself on "dirty"
termination after a mitmproxy crash.
2018-04-17 10:42:40 +12:00
Aldo Cortesi
14aa61b6b1
Merge pull request #3063 from cortesi/sigs
Ignore signal errors on windows + various fixes
2018-04-17 10:18:45 +12:00
Aldo Cortesi
ef4db52887 Ignore signal errors on windows + various fixes
- Ignore the NotImplementedError raised by add_signal_handler on Windows.
- Entrypoints return an integer exit code, or None. Adjust our type annotations
and code to suit.

Fixes #3061
2018-04-17 08:20:52 +12:00
Aldo Cortesi
754cb6cac9
Merge pull request #3048 from cortesi/qbench
Improve benchmarking
2018-04-17 07:35:12 +12:00
Aldo Cortesi
565146311a asyncio: clarify shutdown semantics
This patch clarifies proxy shutdown, and specifies that the master.shutdown()
method is thread-save.
2018-04-16 10:16:51 +12:00
Aldo Cortesi
850c855495 Clean up test/helper_tools directory 2018-04-16 09:20:22 +12:00
Aldo Cortesi
b663a224a3 Improve benchmarking
- The benchmark addon now manages setting up and tearing down the backend and
traffic processes itself.
- Use wrk instead of hey. I get more consistent results with this tool, and hey
shows a strange tail-latency bump that seems artificial.
- Make termination behaviour simpler. The bencmark revealed a bug where .done
events were not called if the proxy was shut down by an addon.
2018-04-16 09:19:14 +12:00
Aldo Cortesi
5f74adc2df
Merge pull request #3059 from obscure76/enh/issue-3053
fix Python 3.6 variable type annotations #3053
2018-04-16 07:35:20 +12:00
oscure76
0e984e1442 fix Python 3.6 variable type annotations #3053 2018-04-14 16:24:41 -07:00
Aldo Cortesi
865a373bd3
Merge pull request #3050 from kira0204/error-script
Handling user script's exceptions, fix #2839
2018-04-15 10:12:44 +12:00
Aldo Cortesi
cabb788072
Merge pull request #3057 from mitmproxy/requires-io-master
[requires.io] dependency update on master branch
2018-04-15 10:03:54 +12:00
Aldo Cortesi
4e126c0fba
Merge pull request #3056 from cortesi/readfile
readfile fixes
2018-04-15 10:03:30 +12:00
requires.io
09e125396c [requires.io] dependency update 2018-04-14 19:10:15 +02:00
Aldo Cortesi
909a02ea03 mitmdump: also set dumper_filter to default filter on startup
Fixes #3051
2018-04-14 13:30:13 +12:00
requires.io
84af021c1b [requires.io] dependency update 2018-04-14 02:56:00 +02:00
Aldo Cortesi
5eb17bbf6d
Merge pull request #3054 from Kriechi/fix-example
fix example filename
2018-04-14 12:15:59 +12:00
Aldo Cortesi
8609de6f31 readfile: add a readfile_filter option
* Add a readfile_filter option that filters flows on read.
* Adjust test suite for asyncio.
* Add asynctest as a dev dependency.
2018-04-14 11:52:02 +12:00
Aldo Cortesi
214498f01c asyncio: adjust readfile.py addon for async 2018-04-14 11:52:02 +12:00
Thomas Kriechbaumer
fe699e555f fix example filename 2018-04-13 09:09:24 +02:00
Maximilian Hils
660aa87a24
fix GA id 2018-04-11 16:58:09 -06:00
kira0204
6780e5025a Handling user script exceptions, fix #2839 2018-04-10 08:33:40 +05:30
Aldo Cortesi
5e2a1ec23c
Merge pull request #3035 from cortesi/aiosimpler
asyncio consolidation
2018-04-07 09:37:58 +12:00
Aldo Cortesi
9dcd15d350 asyncio: ditch the handler context
There are a few reasons for this. First, logs are now async, and can be called
at any time. Second, the event loop is thread local, so there can only ever be
one master per thread. These two things together completely obviate the need
for a handler context.
2018-04-07 09:04:14 +12:00
Aldo Cortesi
6a08ef465f asyncio: remove master.add_log, in favor of a persistent log.Log instance 2018-04-07 09:02:10 +12:00
Aldo Cortesi
ea213782d8 asyncio: remove test master.has_log
Now that logs are async, using this call is almost always a mistake. Signal
this by making it semi-private. The method may go away entirely down the track.
2018-04-07 09:02:10 +12:00
Aldo Cortesi
80f2bac356 asyncio: move log mechanism onto the event loop
Logs are now asynchronous, with a log entry pushed onto the event loop for
handling. To support this, the test mechanism grows an await_log method that
waits for a log entry to appear.
2018-04-07 09:02:10 +12:00
Aldo Cortesi
0fa1280daa asyncio simplify: we don't need a queue for proxy->main loop comms
Instead, we just schedule coroutines directly onto the core loop.
2018-04-07 08:59:08 +12:00
Aldo Cortesi
cdbe6f97af asyncio: remove last vestiage of channel input from master 2018-04-07 08:59:08 +12:00
Aldo Cortesi
506ccc5693
Merge pull request #3031 from itzikBraun/add-httpie-export
added option to export request as httpie command
2018-04-06 10:02:02 +12:00
Aldo Cortesi
4177f50a0f
Merge pull request #2971 from kira0204/user-script-exception
A unified handler for user's scripts exception
2018-04-06 10:01:35 +12:00
Aldo Cortesi
48426d0a20
Merge pull request #3046 from cortesi/backlog
Remove socket listen backlog constraint
2018-04-06 09:59:13 +12:00
Aldo Cortesi
a07415783b Remove socket listen backlog constraint
I have no idea why we did this, but the default value is 128, and setting it
this low drops connections under conditions our users can reasonably be expeted
to reach.
2018-04-06 09:22:01 +12:00
Maximilian Hils
2073df9103
update release instructions 2018-04-05 18:26:55 +02:00
Maximilian Hils
e33cf54e07 update CHANGELOG 2018-04-05 17:10:46 +02:00
Maximilian Hils
74c7699541
Merge pull request #3044 from mhils/issue-3038
fix http retry timeout
2018-04-05 15:45:19 +02:00
Maximilian Hils
50ea3aa420 fix http retry timeout
this fixes #3038
2018-04-05 15:30:04 +02:00
Aldo Cortesi
8e54a365fa
Merge pull request #3032 from mitmproxy/requires-io-master
[requires.io] dependency update on master branch
2018-04-05 07:50:37 +12:00
Aldo Cortesi
780cd989d5
Merge pull request #3029 from cortesi/eventloop
shift core event loop to asyncio
2018-04-04 15:47:01 +12:00
Thomas Kriechbaumer
60a320fde8
Merge pull request #3033 from emilstahl/master
Fixed --no-mouse parameter message
2018-04-03 19:14:55 +02:00
Emil Stahl
8ea8f31e3f
Fixed --no-mouse parameter message
`--no-mouse` is deprecated. Changed parameter to `--set console_mouse=false`
2018-04-03 18:56:12 +02:00
Aldo Cortesi
659fceb697 minor fixes 2018-04-03 10:26:09 +12:00
Aldo Cortesi
7e73e5fa51 asyncio: factor out test server startup wait 2018-04-03 09:46:11 +12:00