This is our first built-in command, which will be used by very many other
commands.
Also add a --commands option to dump all commands, analogous to --options.
- Fix some loading sequence bugs affecting command-line script invocation
- Allow addons to over-ride existing options (with a warning). We need this for
reloading.
- Convert har_dump to new-style arguments, fix and re-instate its test suite.
- Covnert miscelaneous other exmples to new-style args.
Many addons currently save options on configure(), either as individual options
or sometimes by saving the entire options object. The current options should
simply be available on the ctx object, simplifying state management for addons
considerably.
- Remove the watchdog dependency. We now just stat the script file every 2
seconds to check for an updated mtime.
- Further solidify our script testing, and in particular make the example tests
nicer. These should exemplify how we want users to test their own addon
scripts. More work on addon testing to follow.
- Addons now nest, which means that addons can manage addons. This has a number
of salutary effects - the scripts addon no longer has to poke into the global
addons list, we no longer have to replace/remove/boot-outof parent addons when
we load scripts, and this paves the way for making our top-level tools into
addons themselves.
- All addon calls are now wrapped in a safe execution environment where
exceptions are caught, and output to stdout/stderr are intercepted and turned
into logs.
- We no longer support script arguments in sys.argv - creating an option
properly is the only way to pass arguments. This means that all scripts are
always directly controllable from interctive tooling, and that arguments are
type-checked.
For now, I've disabled testing of the har dump example - it needs to be moved
to the new argument handling, and become a class addon. I'll address that in a
separate patch.
- Move default client ciphers - no need to have a 10-line string default.
- Add some style guidance on options help
- Be consistent about reflowing options help
When the path contains dot ".", replacing all dots will generate a non-exist result and raises a FileNotFoundError. Replacing only the last dot fixes this.
While we're here, make checking that there are NO logs on the proxy more
efficient by setting a short timeout. This reduces total test suite running
time by 15% on my system.
This patch implements the lightweight event system I propose in #2144, adds a
custom event "processing_complete" that is triggered after file read, client
replay and server replay, and introduces a KeepServing addon to handle this for
mitmdump.
This patch moves the final pieces of master functionality into addons.
- Add a ReadFile addon to read from file
- Add a separate ReadStdin addon to read from stdin, only used by mitmdump
- Remove all methods that know about io and serialization from master.Master
Attributes on options share a namespace with options themselves. It's getting
too crowded on our Options object, so let's shift some obvious stuff into the
module.
- Add an options parameter to the start() event. This is to be used by addons
on startup to add custom options.
- Add a running() event that is called once the proxy is up and running.
- With the new paradigm we can't log during master __init__, so add a tiny
termstatus addon to print proxy status to terminal once we're running.
name, type, value matches natural language ($x is a bool with a default value of $y)
and also matches the python type annotation order of name: type = value
We now have one option "proxyauth". If this is "any", we accept any
credentials, if it starts with an @ it's treated as a path to an htpasswd file,
if it is of the form username:password it's a single-user credential.
- Regularise presentation and wording
- Help is mandatory for all options
- Auto-generate wording to say that sequence options can be passed multiple
times on the command-line
We now have --set, so only options that really deserve it get a dedicated flag.
I'm inclined to strip this back even more. Feel free to argue that YOUR
favourite option deserves special treatment here.
The --set option is a universal flag for setting options. Some examples:
Turn on a boolean:
mitmdump --set onboarding=false
Add a value to a sequence:
mitumdupm --set setheaders=/foo/bar/voing
Zero a sequence:
mitumdupm --set setheaders
We now have:
--mode regular (the default)
--mode transparent
--mode socks5
--mode reverse:SPEC
--mode upstream:SPEC
Where SPEC is a host specification.
Start dealing with corner cases:
- Sizes are always stored in options as strings
- Add a new core addon that's responsible for verifying settings that don't
belong to an addon
- Add a _processed scratch space on the Options object for processed core
values to be stored in. This is pretty dirty, but less dirty than re-parsing
values every time. We'll come up with something better down the track.
This commit:
- Adds a help field to options
- Adds a function to generate parser definitions from options
- Uses this to migrate all boolean flags over to the new system
- Makes all booleans consistently follow the --foo/--not-foo convention
There are a number of things left to be done here:
- Argparse doesn't give us a nice way to format --foo --not-foo help. Click
does, and moving to click is a goal down the track.
- For now, we remove all short aliases. I want to re-evaluate these
systematically once we have the new structure in place.
