Grasscutters/mitmproxy
3
0
Fork 0
mirror of https://github.com/Grasscutters/mitmproxy.git synced 2024-11-25 18:03:50 +00:00
Code Issues Projects Releases Wiki Activity
8,267 Commits 1 Branch 67 Tags 50 MiB
642de3925d
Commit Graph

312 Commits

Author SHA1 Message Date
Maximilian Hils
eda1b39a74 minor fixes 2016-10-25 22:06:52 -07:00
Slobodan Mišković
39d7ba852c Include boudary=... in mutipart postData 
While the HAR spec is not very explicit and their example shows just this one example: ```json
"postData": {
    "mimeType": "multipart/form-data"
}
```
Would it not make sense to include all the information necessary to parse out the post data `text`. Eg.
```json
"postData": {
           "text": "--xYzZY\r\nContent-Disposition: form-data; name=\"sort1\"\r\n\r\noldest date first\r\n--xYzZY--\r\n",
           "mimeType": "multipart/form-data; boundary=xYzZY"
         },
```
Currently, full mimeType is included only in `content-type` request header.

Elsewhere in HAR spec they include the 'extras', eg ```json
"content": {
    "mimeType": "text/html; charset=utf-8"
}
``` 
So one could argue that `mimeType` should include all information necessary to interpret the data. In case of `multipart/form-data`, as per RFC2046 http://www.ietf.org/rfc/rfc2046.txt
```
 The Content-Type field for multipart entities requires one parameter, "boundary".
```
I believe that earlier incarnations, eg `har_exporter.py` included it in the mimeType.
 2016-10-24 14:34:04 -07:00
Maximilian Hils
a1a792aeac various encoding fixes, fix #1650 2016-10-22 18:47:12 -07:00
Slobodan Mišković
0526d94f4a Handle bytes in request parameters 2016-10-22 18:28:32 -07:00
Brady Law
0d0a3a51df The first argument should be the filter, then the flow. 2016-10-21 16:47:19 -07:00
Aldo Cortesi
8430f857b5 The final piece: netlib -> mitproxy.net 2016-10-20 11:56:38 +13:00
Aldo Cortesi
f45f4e677e netlib.strutils -> mitmproxy.utils.strutils 2016-10-20 10:11:58 +13:00
Aldo Cortesi
7440232f60 netlib.version -> mitmproxy.version 2016-10-20 09:20:44 +13:00
Aldo Cortesi
e73c7fe77e mitmproxy.protocol -> mitmproxy.proxy.protocol 
The protocols here are compltely proxy-specific, are only used from within the
proxy module, and are not exposed to users.
 2016-10-19 23:11:56 +13:00
Aldo Cortesi
24cf8da27e Move all tools into mitmproxy.tools, move models/* to top level 
The primary motivation here (and for all the other moving around) is to present
a clean "front of house" to library users, and to migrate primary objects to
the top of the module hierarchy.
 2016-10-19 20:26:05 +13:00
Aldo Cortesi
5a68d21e8c Remove flow module entirely, move contents to top level 
mitmproxy.flow.io -> mitmproxy.io
mitmproxy.flow.export -> mitmproxy.export
 2016-10-19 15:08:35 +13:00
Aldo Cortesi
7c32d4ea2a flow.state -> addons.state 2016-10-19 14:48:42 +13:00
Aldo Cortesi
22eebfd574 addons.Addons -> addonmanager, builtins -> addons 2016-10-19 14:39:39 +13:00
Aldo Cortesi
bce387a5a0 Kill flow.master - create master.Master 
Also extract events into .events
 2016-10-19 13:22:50 +13:00
Aldo Cortesi
87629586ae web app cleanups: tests and examples 2016-10-19 11:48:51 +13:00
Maximilian Hils
02d3d61820 fix redirect_requests.py example 2016-10-16 23:49:54 -07:00
Aldo Cortesi
c774a9fec9 python3: clean up super and __future__ 2016-10-17 18:03:07 +13:00
Aldo Cortesi
a647b30365 python3: clean up class brackets 2016-10-17 17:37:08 +13:00
Aldo Cortesi
fb22f2ff4f Zap object base class 2016-10-17 17:37:08 +13:00
Aldo Cortesi
ce98a9219e test & examples: zap six 2016-10-17 16:45:45 +13:00
Aldo Cortesi
55cb2a8547 docs: logging and the context 2016-10-16 20:26:06 +13:00
Aldo Cortesi
fb69c9c345 docs: overview, classes, arguments 2016-10-16 20:26:06 +13:00
Thomas Kriechbaumer
bb60b76af4 use flowfilter.match 2016-10-03 11:45:54 +02:00
Thomas Kriechbaumer
ba84248910 rename mitmproxy.filt -> mitmproxy.flowfilter 2016-10-03 11:45:54 +02:00
phackt
8021427ab9 Fixes - #1555 sslstrip.py flow.response.headers (#1556) 
* Fixes - #1555 sslstrip.py flow.response.headers

* #1557 - add enhancements in inline script sslstrip.py with upgrade-insecure-requests stripping

* #1557 - update to match python style guide

* #1555, #1556, update to a bytes pattern
 2016-09-25 19:29:26 -07:00
smill
7cd8456445 Added a description to the shim loader, and renamed it. 2016-09-14 19:25:53 +00:00
smill
fbfedbdc8f Improved error-handling / supplemented documention. 2016-09-04 01:30:27 +00:00
Aldo Cortesi
afe34e8b28 Improve the way we handle upstream errors 
- Don't log a traceback for either HTTP or HTTPS DNS resolution or TCP
connection errors. These are "ordinary" errors, not mitmproxy issues.
- Ensure that the error handler is correctly called for SSL-related protocol
errors.
 2016-09-01 12:32:09 +12:00
Aldo Cortesi
9306e80e65 Adjust flowbasic example for Options API changes 2016-09-01 12:11:00 +12:00
Maximilian Hils
f27028f58e introduce Response.make for simple response creation 2016-08-23 00:17:06 -07:00
Shadab Zafar
bf4425de80 Fix issue with binary content in json 2016-08-15 12:00:23 +05:30
Shadab Zafar
4f1fb43dcc Use postData field in PUT, PATCH requests too 
The HAR spec isn't really clear on whether this should be the case,
but Google Chrome does this, so I think we should too.
 2016-08-15 12:00:23 +05:30
Shadab Zafar
15c488225f Refactor format_cookies 2016-08-15 12:00:23 +05:30
Shadab Zafar
567cbe6cb9 Support .zhar compression 2016-08-15 12:00:23 +05:30
Shadab Zafar
ac97e5efa1 Add text field to response content 2016-08-15 12:00:23 +05:30
Shadab Zafar
9aa230707d Add serverIPAddress field 2016-08-15 12:00:23 +05:30
Shadab Zafar
7de48fc197 Add postData field 2016-08-15 12:00:23 +05:30
Shadab Zafar
b14eb57db1 Add SSL & Connect timings 2016-08-15 12:00:23 +05:30
Shadab Zafar
23b9ef799e Add a ctx.log on finish 2016-08-15 12:00:23 +05:30
Shadab Zafar
456f80d862 Open JSON file in text mode 2016-08-15 12:00:23 +05:30
Shadab Zafar
2c9240fd22 Simplify name_value 2016-08-15 12:00:23 +05:30
Shadab Zafar
e9c6563367 Fix wrong import 2016-08-15 12:00:23 +05:30
Shadab Zafar
a2a8283fa4 Improve cookies formatting 2016-08-15 12:00:23 +05:30
Shadab Zafar
55f1ffe0b1 Format Cookies according to the HAR Spec 2016-08-15 12:00:23 +05:30
Shadab Zafar
a0932af55c Remove pages object 
The HAR Spec says that the field can be left out by applications that don't
group by pages.

http://www.softwareishard.com/blog/har-12-spec/#log
 2016-08-15 12:00:23 +05:30
Shadab Zafar
250e4a17d0 Welcome har_dump 2016-08-15 12:00:23 +05:30
Shadab Zafar
8ba34be7ab Goodbye har_extractor 2016-08-15 12:00:23 +05:30
Maximilian Hils
fb238ad63e make the linter happy 2016-07-23 14:35:04 -07:00
Maximilian Hils
b78fea3784 add remote debug example 2016-07-23 14:02:54 -07:00
Aldo Cortesi
afda175e1c Addon iface: .configure(options) -> .configure(options, updated) 2016-07-23 15:43:55 +12:00
Aldo Cortesi
dbafe9f87b scripts: refactor some examples that keep global state 
We now have a better way to do this.
 2016-07-23 12:24:09 +12:00
Maximilian Hils
3602fd7a36 Merge remote-tracking branch 'origin/master' into message-body-encoding 2016-07-15 23:17:57 -07:00
Aldo Cortesi
917d51bd22 Fix HAR extractor 
Thanks to @mhils
 2016-07-15 15:01:35 +12:00
Aldo Cortesi
92a5076bd0 Adjust concurrent tests for start method 2016-07-15 15:00:37 +12:00
Aldo Cortesi
4ba7ce50c6 Add .start for addons and scripts 
Also improve error messages, fix various unit tests
 2016-07-15 14:52:34 +12:00
Aldo Cortesi
b94f5fd361 Convert examples and example tests for new-style scripts 
Remove the test that just loads all the example scripts for now - it's a very
low-value test, and we need to think of something better.
 2016-07-14 19:54:15 +12:00
Maximilian Hils
5d2b7c52f9 move script context to mitmproxy.ctx 2016-07-08 19:57:57 -07:00
Maximilian Hils
7c67faa8da remove script contexts 2016-07-07 23:50:55 -07:00
Maximilian Hils
c048ae1d5b remove context from all scripts 2016-07-07 20:14:52 -07:00
Maximilian Hils
76473d44e0 py3++ 2016-07-07 17:50:26 -07:00
Maximilian Hils
f259b9478f disable harparser on py3 2016-07-07 01:21:15 -07:00
Maximilian Hils
444f0a4c39 py3++ 2016-07-06 17:31:08 -07:00
Maximilian Hils
d51cf543bb remove clean_bin, clarify unicode handling 2016-07-05 15:11:32 -07:00
Maximilian Hils
dbf7cb1a44 update examples: no decoded() anymore 🎉 2016-07-02 02:01:46 -07:00
Tai Dickerson
c2d0910e59 Make har_extractor.py output HAR 1.2 spec-compliant 
HAR files were failing to load in harviewer (http://www.softwareishard.com/har/viewer/) due to:
- ISO 8601 dates for startedDateTime missing timezone (http://www.softwareishard.com/blog/har-12-spec/#pages)
-- Used UTC but could add detection of default system timezone if desired
- pages object missing pageTimings (http://www.softwareishard.com/blog/har-12-spec/#pageTimings)
-- Used {} because all child fields are optional
 2016-06-14 12:47:10 -04:00
Maximilian Hils
63c6660895 update examples, tests, docs 2016-06-13 18:17:09 -07:00
Aldo Cortesi
b3bf754e53 Simplify script concurrency helpers 
We now have take() to prevent double-replies.
 2016-06-08 11:21:38 +12:00
Aldo Cortesi
a388ddfd78 A new interface for reply 
Reply is now explicit - it's no longer a callable itself. Instead, we have:

    reply.kill()            - kill the flow
    reply.ack()             - ack, but don't send anything
    reply.send(message)     - send a response

This is part of an incremental move to detach reply from our flow objects,
and unify the script and handler interfaces.
 2016-06-08 10:44:20 +12:00
Aldo Cortesi
09da1febbd Shift a bunch more string-related functions to strutils 2016-06-02 13:04:19 +12:00
Aldo Cortesi
cccdc98426 Utils reorganisation: add netlib.strutils 
Extract a number of string and format-related functions to netlib.strutils.
 2016-06-02 12:31:41 +12:00
Maximilian Hils
2e633af5aa handle_* -> * leftovers 2016-05-29 13:01:45 -07:00
Thomas Kriechbaumer
8eeab37a07 more style cleanup 
Use this to check:
flake8 --count mitmproxy netlib pathod examples test
 2016-05-29 11:55:12 +02:00
Thomas Kriechbaumer
0c2c017727 change comment indentation 2016-05-29 11:04:55 +02:00
Thomas Kriechbaumer
c707479bcd Merge pull request #1173 from mitmproxy/format-examples 
Format examples
 2016-05-29 11:03:54 +02:00
Aldo Cortesi
27fff86cf7 Update method names in /examples/flowbasic 
Please enter the commit message for your changes. Lines starting
 2016-05-29 20:51:30 +12:00
Maximilian Hils
17382b963e format examples 2016-05-29 01:23:39 -07:00
Aldo Cortesi
0176f50e4f Adapt examples 2016-05-29 12:54:52 +12:00
Maximilian Hils
6f8db2d7eb improve MultiDict, add ImmutableMultiDict, adjust response.cookies 2016-05-18 22:50:19 -07:00
Maximilian Hils
44ac64aa72 add MultiDict 
This commit introduces MultiDict, a multi-dictionary similar to
ODict, but with improved semantics (as in the Headers class).
MultiDict fixes a few issues that were present in the Request/Response
API. In particular, `request.cookies["foo"] = "bar"` has previously been a
no-op, as the cookies property returned a mutable _copy_ of the cookies.
 2016-05-18 18:46:42 -07:00
Jason Pepas
b174fd5e1f Adding example which turns every reponse into an HTTP 500. (#1136) 
* Adding example which turns every reponse into an HTTP 500.

* Prefer b"" over None for response.content.
 2016-05-11 15:26:17 -06:00
Maximilian Hils
909d5ec87e FlowReadError -> FlowReadException 2016-04-29 11:17:49 -07:00
Maximilian Hils
bc60c26c7b py3++ 2016-04-13 22:34:28 -07:00
Maximilian Hils
35a6d6e7f6 response.msg -> response.reason 2016-04-02 22:54:46 +02:00
Maximilian Hils
6f902ffbb3 py3++: iteritems 2016-03-20 19:56:22 +01:00
Maximilian Hils
88d365cfe6 py3++: cStringIO 2016-03-20 19:40:03 +01:00
cle1000
0a478e4e39 Update modify_response_body.py 
fix typo
 2016-03-15 15:36:10 +01:00
Shadab Zafar
7c5791e51a Use the contextmanager in all tests 2016-03-11 17:57:57 +05:30
Shadab Zafar
2cd5392657 Add test for iframe_injector example 2016-03-11 17:57:57 +05:30
Shadab Zafar
9ef1522b5f Update modify_querystring example & add test for it 2016-03-11 17:57:57 +05:30
Shadab Zafar
106b296871 Update modify_form example to not use deprecated functions 2016-03-11 17:57:57 +05:30
Maximilian Hils
167bd77683 fix content view example inline script 2016-03-10 21:17:05 +01:00
Shadab Zafar
7108d72705 Fix pprint import 2016-03-04 03:02:49 +05:30
Shadab Zafar
1c80599379 Use default empty page_list 2016-03-04 03:02:33 +05:30
Shadab Zafar
c982f579dd Add basic test for har_extractor 2016-03-04 02:32:01 +05:30
Shadab Zafar
1f41719bbc Remove some duplication and make more pythonic har_extractor 2016-03-02 11:39:48 +05:30
Shadab Zafar
27faea2355 Fixup HAR Extractor 2016-03-01 20:20:18 +05:30
Shadab Zafar
6bb27c162f har_extractor: Replace deprecated stuff with recommended ones 2016-02-20 21:01:59 +05:30
Shadab Zafar
be9329dfd4 har_extractor: Use items() instead of iteritems() 2016-02-20 21:01:27 +05:30
Aldo Cortesi
36e070ee25 Pull bin to the top, move pathod examples to examples dir 2016-02-18 10:15:59 +13:00
Aldo Cortesi
92597f82ea Docs and examples to top level 2016-02-18 09:27:08 +13:00
Maximilian Hils
33fa49277a move mitmproxy 2016-02-15 14:58:46 +01:00
Maximilian Hils
0f54f2990c Merge pull request #872 from bazzinotti/inline-tcp 
Add "tcp_message" Inline Script Hook
 2016-01-17 19:54:59 +01:00
Niko Kommenda
55e89865ff no longer strips Accept-Encoding as mitmproxy can handle compression 2016-01-12 22:25:42 +00:00
Niko Kommenda
25705af76d added sslstrip to inline script examples 2016-01-12 16:41:41 +00:00
Michael J. Bazzinotti
e766838a1a Add Inline Script Hooks to TCP mode 2016-01-10 13:05:01 -05:00
Stefan Wärting
27b801ebe2 Update redirect_requests.py 
replace http version with "HTTP/1.1"
 2016-01-08 16:02:27 +01:00
Maximilian Hils
247f27d821 minor stylistic fixes 2015-11-14 03:46:34 +01:00
Chris Czub
e72a9a62a1 Feedback from PR #832 2015-11-13 16:55:27 -05:00
Maximilian Hils
b13acd7956 adjust to netlib request changes + docs 2015-09-26 01:23:59 +02:00
Maximilian Hils
906360f63f code -> status_code, httpversion -> http_version 2015-09-17 15:16:33 +02:00
Maximilian Hils
14a5f405fd use new netlib exceptions 2015-09-17 02:13:28 +02:00
Maximilian Hils
0af0608978 adjust to netlib changes 2015-09-16 18:45:22 +02:00
Maximilian Hils
e63ef3a520 add inline script for conditional tls passthrough 
fix #646
 2015-09-08 21:35:15 +02:00
Maximilian Hils
5125c669cc adjust to new netlib Headers class 2015-09-05 20:45:58 +02:00
Maximilian Hils
a18db90ea3 update example inline scripts 2015-09-04 00:46:42 +02:00
Maximilian Hils
99126f62ed remove depth attribute from set_server 2015-09-03 18:25:36 +02:00
Maximilian Hils
b04e6e56ab update inline script hooks 2015-08-31 17:05:52 +02:00
Maximilian Hils
7450bef615 fix dns_spoofing example, avoid connecting to itself 2015-08-31 13:43:30 +02:00
Maximilian Hils
1e9aef5b1e fix upstream proxy server change, update example 2015-08-31 00:14:42 +02:00
Maximilian Hils
3873e08339 remove old code 2015-08-30 03:42:11 +02:00
FreeArtMan
4d9bf45e82 Fixing issue #710. Prety printing in read_dumpfile 2015-08-19 09:59:13 +01:00
Aldo Cortesi
9286f5b5b4 Small tweaks to read_dumpfile example. 2015-08-17 11:29:24 +12:00
Thomas Kriechbaumer
891fa50e55 move code to netlib 2015-08-10 21:04:17 +02:00
Thomas Kriechbaumer
4f38c6b90e attach application protocol to connection 2015-07-30 19:53:29 +02:00
Thomas Kriechbaumer
a5d9e1f44d move code to netlib and implement protocols 2015-07-30 13:53:17 +02:00
Jim Shaver
b51363b3ca Merge remote-tracking branch 'upstream/master' into print-bracket-fix 
Conflicts:
	examples/har_extractor.py
	examples/nonblocking.py
	examples/read_dumpfile
	libmproxy/web/app.py
 2015-05-31 01:21:44 -04:00
Jim Shaver
4fe2c069cc Fixed print function to be inline with python 3 2015-05-29 23:17:48 -04:00
Aldo Cortesi
a05a70d816 Add coding style check, reformat. 2015-05-30 12:03:28 +12:00
Aldo Cortesi
937a358aa9 Merge branch 'master' of ssh.github.com:mitmproxy/mitmproxy 2015-04-14 12:01:37 +12:00
Aldo Cortesi
bea0bd236a Housekeeping and cleanups 
- No output to stdout on load in examples - they muck up the test suite.
- Use the odict module directly, rather than aliasing it. The small convenience
this gives to scripters is not worth it.
- Move the cookie tests from the flow test module to the protocol_http test
module.
 2015-04-14 11:58:10 +12:00
Maximilian Hils
ba149d90ad spaces, not tabs 2015-04-12 03:50:14 +02:00
Maximilian Hils
c0a318566a add flowwriter example 2015-04-12 03:47:58 +02:00
Maximilian Hils
0aa2fca25a Merge pull request #544 from ryoqun/websocket-connection-header-case-sensitivity 
Make the Websocket's connection header value case-insensitive
 2015-04-08 01:23:54 +02:00
Maximilian Hils
74aff39b28 add example which uses filt 2015-04-08 00:21:49 +02:00
Ryo Onodera
9674a0869c Make the Websocket's connection header value case-insensitive 2015-03-31 16:07:04 +09:00
Maximilian Hils
e1b6cf9401 fix #319 2015-02-27 15:24:27 +01:00
Maximilian Hils
8d975e80ff Merge remote-tracking branch 'sakurai/feature/issue319' 2015-02-27 14:47:50 +01:00
Maximilian Hils
595bde2202 add example inline script for dns spoofing, refs #486 2015-02-27 12:55:16 +01:00
Youhei Sakurai
1d42c1b3c4 Modify example to notify incompatibility with --stream SIZE command line 
option; https://github.com/mitmproxy/mitmproxy/issues/319
 2015-02-27 03:17:40 +09:00
Youhei Sakurai
77bb0b74ed Maybe it should work; https://github.com/mitmproxy/mitmproxy/issues/319 2015-02-27 02:44:47 +09:00
Aldo Cortesi
bb5fb2dbe0 Clean up har_extractor example 
- Fix spacing, line length, unused imports, unusual import idioms
- Prevent it from barfing into our test output
 2015-01-02 13:42:43 +13:00
Maximilian Hils
a7a9ef826c fix tests 2014-12-01 03:36:04 +01:00
Maximilian Hils
40fbb95701 Merge branch 'master' of github.com:mitmproxy/mitmproxy 2014-12-01 03:05:11 +01:00
Maximilian Hils
5b1fefee9b add inline script example for websocket passthrough, fix #340 2014-12-01 03:04:48 +01:00
Gabriel Kirkpatrick
cf7404cfe0 Changed argument name from confdir to cadir in flowbasic example 2014-11-28 12:36:31 -05:00
Justus Wingert
ddce662fe6 Added try/except block for import errors with harparser and pytz. 2014-11-15 22:39:15 +01:00
Justus Wingert
c84ad384f6 Updated setup.py and moved requirements to examples section. Included examples section in requirements.txt. Updated har_extractor to use command line arguments. 2014-11-15 22:37:32 +01:00
Justus Wingert
a7ab06d80e Switched to pytz. Added comment for clarification on behaviour of HAREncodable. Added missing parameter in reset(). Fixed accessing headers. 2014-11-15 22:04:52 +01:00
Justus Wingert
4227feef37 It seems get_decoded_content can actually be shorter than content due to encoding issues. Since I'm not crazy after all it seems safe to push. 2014-11-15 21:14:50 +01:00
Justus Wingert
4342d79dc0 Removed the globals and replaced them with internal attributes of _HARLog. Minor bugfixes to make ssl timings work. 2014-11-15 20:11:25 +01:00