Commit Graph

252 Commits

Author SHA1 Message Date
Aldo Cortesi
e6eeab6094 Revamp how addons work
- Addons now nest, which means that addons can manage addons. This has a number
of salutary effects - the scripts addon no longer has to poke into the global
addons list, we no longer have to replace/remove/boot-outof parent addons when
we load scripts, and this paves the way for making our top-level tools into
addons themselves.
- All addon calls are now wrapped in a safe execution environment where
exceptions are caught, and output to stdout/stderr are intercepted and turned
into logs.
- We no longer support script arguments in sys.argv - creating an option
properly is the only way to pass arguments. This means that all scripts are
always directly controllable from interctive tooling, and that arguments are
type-checked.

For now, I've disabled testing of the har dump example - it needs to be moved
to the new argument handling, and become a class addon. I'll address that in a
separate patch.
2017-04-25 22:13:44 +12:00
Maximilian Hils
482c1579f2 proxapp -> proxapp.local
Chrome doesn't like it if there's no TLD.
2017-03-26 15:02:47 +02:00
Aldo Cortesi
65f0885bd6 addon loader: add boot_into, which replaces returning from start()
While we're here, expand test coverage for addonmanager to 100%, and promote to
individual coverage.
2017-03-25 10:48:12 +13:00
Aldo Cortesi
541c1e8b9f addons: start -> load throughout 2017-03-25 10:48:12 +13:00
Aldo Cortesi
1410cbb4b6 Remove test handlers by using taddons.RecordingMaster
This also means expanding and tweaking the recording master API, which we
reflect through the current test suite
2017-03-16 18:05:57 +13:00
Aldo Cortesi
b745428b5c Enable custom options in config files
We also now ignore unknown options in config files by default, and print a
warning if verbosity is incremented.
2017-03-14 09:40:33 +13:00
Aldo Cortesi
c24f7d8e12 Optmanager: handle unknown options in value sets 2017-03-14 08:32:19 +13:00
Aldo Cortesi
0c6663d0d5 Enable custom options for addons
- Add an options parameter to the start() event. This is to be used by addons
on startup to add custom options.
- Add a running() event that is called once the proxy is up and running.
- With the new paradigm we can't log during master __init__, so add a tiny
termstatus addon to print proxy status to terminal once we're running.
2017-03-14 08:32:19 +13:00
Maximilian Hils
927b5707fe fix tcp.Address leftovers
this fixes the issue described in https://github.com/mitmproxy/mitmproxy/issues/2119#issuecomment-285067292
2017-03-08 16:18:34 +01:00
Nikhil Soni
0081d9b828 Merge branch 'master' into on-issues 2017-03-03 12:58:44 +05:30
Nikhil Soni
317d183ba4 Changes dns_spoofing example to use --keep-host-header 2017-03-03 12:34:36 +05:30
David Dworken
99b584ad7d added XSS scanner example (#1907) 2017-02-27 18:22:39 +01:00
Ujjwal Verma
e723a58af5 Remove pytz in hardump 2017-02-25 23:07:47 +05:30
Thomas Kriechbaumer
36352c9539 protobuf: coverage++ 2017-02-19 14:29:09 +01:00
Maximilian Hils
b9e31f213f .headers["host"] -> .host_header 2017-02-18 00:13:14 +01:00
Maximilian Hils
5acdd78b15 fix typo 2017-02-14 17:45:54 +01:00
Ujjwal Verma
c622622c59 Encoding fixes and tests 2017-02-05 18:59:01 +05:30
Ammonite
aaff9dfd32 Reset the host header to the correct destination 2017-01-30 23:41:47 +01:00
Ammonite
c2c6050df3 Store original host in flow metadata 2017-01-29 14:33:53 +01:00
Ammonite
0ca1916f1b Fix host extraction 2017-01-22 15:28:14 +01:00
Ammonite
e8fc4af4c6 Follow PEP-8 and add comment 2017-01-22 14:59:46 +01:00
Ammonite
93172460aa Add blank lines for lint 2017-01-21 09:39:34 +01:00
Ammonite
8aa250d679 Change class name 2017-01-20 23:48:26 +01:00
Ammonite
a55eba3b37 Get the the original header in requestheaders instead of request 2017-01-20 23:43:53 +01:00
Feei
95cca4ce75 update stream.py 2017-01-16 18:30:05 +08:00
Feei
eaaec4353d update log_events.py 2017-01-16 18:11:18 +08:00
Maximilian Hils
98a079aa69 rename logging.py example. this conflicted with Python's stdlib 2016-12-19 16:21:17 +01:00
Maximilian Hils
c4e9000021 fix #1858 2016-12-19 01:15:10 +01:00
Maximilian Hils
3e37cbd061 minor fixes 2016-12-10 12:06:33 +01:00
Maximilian Hils
293b79af91 remove lxml-dependent code 2016-12-10 10:19:05 +01:00
Maximilian Hils
45332006a3 mitmweb: 100% app test coverage, numerous fixes 2016-11-23 22:35:07 +01:00
Maximilian Hils
40f0193dda remove mitmweb auth
9b08279c7c removed the actual functionality.
we should not have a command line switch that does nothing. :)
2016-11-22 18:27:16 +01:00
Maximilian Hils
c90405253a remove stickycookie example
The recommended way to do this is mitmproxy/addons/stickycookie.py
2016-11-21 02:46:25 +01:00
Maximilian Hils
9af8f4bb31 organize examples
This commit is largely based on work by Thiago Arrais (@thiagoarrais)
and Shane Bradfield (@l33tLumberjack). I wasn't really able to get their
PR reasonably merged onto the latest master, so I reapplied their changes
manually here and did some further improvements on that.
2016-11-21 02:28:10 +01:00
Mike Fotinakis
d16a3753d7 Remove dead run_scripthooks example reference. 2016-11-17 16:33:51 -08:00
Maximilian Hils
2a2387fb32 explain host=pretty_host assignment 2016-11-09 13:11:31 +01:00
phackt
dc44465c92 fix SNI for transparent mode - #1638
In transparent mode host is set with the target server ip.
Attribute flow.request.host is used as SNI while mitmproxy is
initiating TLS handshake, so it should be set with the pretty_host.
2016-11-08 15:39:24 +01:00
Aldo Cortesi
4eb2b56dec Let's not over-ride __bool__ on connection objects
If I had a thousand years and every thesaurus in the world, I still couldn't
adequately express how much I dislike this piece of interface design.
2016-11-04 10:59:41 +13:00
Thomas Kriechbaumer
d56bbb95e2 rename logging .py to prevent import errors 2016-10-29 11:31:38 -07:00
Maximilian Hils
f26a375560 fix #1678 2016-10-27 12:55:39 -07:00
Maximilian Hils
eda1b39a74 minor fixes 2016-10-25 22:06:52 -07:00
Slobodan Mišković
39d7ba852c Include boudary=... in mutipart postData
While the HAR spec is not very explicit and their example shows just this one example: ```json
"postData": {
    "mimeType": "multipart/form-data"
}
```
Would it not make sense to include all the information necessary to parse out the post data `text`. Eg.
```json
"postData": {
           "text": "--xYzZY\r\nContent-Disposition: form-data; name=\"sort1\"\r\n\r\noldest date first\r\n--xYzZY--\r\n",
           "mimeType": "multipart/form-data; boundary=xYzZY"
         },
```
Currently, full mimeType is included only in `content-type` request header.

Elsewhere in HAR spec they include the 'extras', eg ```json
"content": {
    "mimeType": "text/html; charset=utf-8"
}
``` 
So one could argue that `mimeType` should include all information necessary to interpret the data. In case of `multipart/form-data`, as per RFC2046 http://www.ietf.org/rfc/rfc2046.txt
```
 The Content-Type field for multipart entities requires one parameter, "boundary".
```
I believe that earlier incarnations, eg `har_exporter.py` included it in the mimeType.
2016-10-24 14:34:04 -07:00
Maximilian Hils
a1a792aeac various encoding fixes, fix #1650 2016-10-22 18:47:12 -07:00
Slobodan Mišković
0526d94f4a Handle bytes in request parameters 2016-10-22 18:28:32 -07:00
Brady Law
0d0a3a51df The first argument should be the filter, then the flow. 2016-10-21 16:47:19 -07:00
Aldo Cortesi
8430f857b5 The final piece: netlib -> mitproxy.net 2016-10-20 11:56:38 +13:00
Aldo Cortesi
f45f4e677e netlib.strutils -> mitmproxy.utils.strutils 2016-10-20 10:11:58 +13:00
Aldo Cortesi
7440232f60 netlib.version -> mitmproxy.version 2016-10-20 09:20:44 +13:00
Aldo Cortesi
e73c7fe77e mitmproxy.protocol -> mitmproxy.proxy.protocol
The protocols here are compltely proxy-specific, are only used from within the
proxy module, and are not exposed to users.
2016-10-19 23:11:56 +13:00
Aldo Cortesi
24cf8da27e Move all tools into mitmproxy.tools, move models/* to top level
The primary motivation here (and for all the other moving around) is to present
a clean "front of house" to library users, and to migrate primary objects to
the top of the module hierarchy.
2016-10-19 20:26:05 +13:00