Commit Graph

1589 Commits

Author SHA1 Message Date
Maximilian Hils
61afac8637 Merge remote-tracking branch 'Kriechi/proxy-refactor' into proxy-refactor
Conflicts:
	libmproxy/protocol2/http_proxy.py
	libmproxy/protocol2/tls.py
	libmproxy/proxy/connection.py
2015-08-16 12:47:13 +02:00
Maximilian Hils
4c31ffd90f minor fixes 2015-08-16 12:43:15 +02:00
Thomas Kriechbaumer
1e40d34e94 add ALPN to proxy connections 2015-08-16 11:36:31 +02:00
Maximilian Hils
a175572447 Merge remote-tracking branch 'Kriechi/proxy-refactor' into proxy-refactor 2015-08-15 20:22:45 +02:00
Maximilian Hils
2a15479cdb fix bugs, make https work 2015-08-15 20:20:46 +02:00
Thomas Kriechbaumer
a9dd82c986 add ALPN to proxy connections 2015-08-15 17:49:40 +02:00
Maximilian Hils
0dd243c5e4 various fixes 2015-08-15 16:26:12 +02:00
Maximilian Hils
747699b126 more work on http protocol 2015-08-14 16:49:52 +02:00
Maximilian Hils
808218f4bc more work on http layer 2015-08-14 10:41:11 +02:00
Maximilian Hils
aef3b626a7 wip commit 2015-08-11 20:32:14 +02:00
Maximilian Hils
026330a3b0 cleaner Exceptions, ssl -> tls, upstream proxy mode 2015-08-11 20:32:13 +02:00
Maximilian Hils
314e0f5839 add reverseproxy mode, fix bugs 2015-08-11 20:32:12 +02:00
Maximilian Hils
aac0ab23eb simplify layer code, add yield_from_callback decorator 2015-08-11 20:32:12 +02:00
Maximilian Hils
c46e3f90bb apply fixes from proxy-refactor-cb branch 2015-08-11 20:32:11 +02:00
Maximilian Hils
531ca4a356 minor fixes 2015-08-11 20:32:10 +02:00
Maximilian Hils
e815915b22 add auto layer, multiple other fixes 2015-08-11 20:32:10 +02:00
Maximilian Hils
c1d016823c move files around 2015-08-11 20:32:09 +02:00
Maximilian Hils
be995ddbd6 add ssl layer 2015-08-11 20:32:09 +02:00
Maximilian Hils
863113f989 first initial proof-of-concept 2015-08-11 20:32:08 +02:00
Thomas Kriechbaumer
891fa50e55 move code to netlib 2015-08-10 21:04:17 +02:00
Maximilian Hils
adf2877020 fix #705 2015-08-03 18:07:30 +02:00
Thomas Kriechbaumer
6cce2f14d7 fix tests 2015-08-02 11:26:33 +02:00
Thomas Kriechbaumer
ff1e7e6409 fix imports for encoding 2015-08-01 14:53:34 +02:00
Thomas Kriechbaumer
b732a1cb3d reimplement streaming for HTTP/1 2015-08-01 14:37:33 +02:00
Thomas Kriechbaumer
54866c0faa revert hard-coded HTTP/2 protocol changes 2015-08-01 14:37:33 +02:00
Thomas Kriechbaumer
db02553e2a move code from mitmproxy to netlib 2015-08-01 14:37:32 +02:00
Thomas Kriechbaumer
8a05151170 revert ALPN-related changes to fix master
In the future this should be added again to support HTTP/2.
2015-08-01 14:37:32 +02:00
Thomas Kriechbaumer
4f38c6b90e attach application protocol to connection 2015-07-30 19:53:29 +02:00
Thomas Kriechbaumer
a5d9e1f44d move code to netlib and implement protocols 2015-07-30 13:53:17 +02:00
Thomas Kriechbaumer
89f22f7359 refactor connection & protocol handling 2015-07-30 13:53:17 +02:00
Maximilian Hils
d2ae6b630e Merge pull request #548 from macmantrl/no_serverconn
Prevent unnecessary upstream server connects
2015-07-26 11:58:21 +02:00
Aldo Cortesi
ee163ed5b0 Bump version for dev 2015-07-26 17:46:49 +12:00
Thomas Kriechbaumer
e9bbcfadc1 fix module imports 2015-07-25 20:00:34 +02:00
Maximilian Hils
0892649d31 Merge pull request #675 from Kriechi/protocol-refactor
HTTP protocol refactoring
2015-07-24 17:52:44 +02:00
Maximilian Hils
7bf8088d80 improve inline scripts, fix #683, fix #684 2015-07-24 02:57:56 +02:00
Maximilian Hils
9b3fe80697 minor style improvements 2015-07-22 16:00:32 +02:00
Thomas Kriechbaumer
cedc0407de refactor to use netlib.http protocols 2015-07-22 15:30:12 +02:00
Thomas Kriechbaumer
bb265d0c40 use new netlib module names 2015-07-22 15:30:11 +02:00
Thomas Kriechbaumer
df1b0df39f use netlib.http_semantics for generic data 2015-07-22 15:30:11 +02:00
Maximilian Hils
170568ded0 fix text selection hint 2015-07-22 00:41:02 +02:00
Maximilian Hils
cacf767b5c bump version 2015-07-22 00:16:38 +02:00
Maximilian Hils
822bd10465 Merge pull request #677 from isra17/json_error
Use `ensure_ascii` in wep.app#broadcast
2015-07-14 00:53:32 +02:00
isra17
c33df9dd1c Use ensure_ascii in wep.app#broadcast
Otherwise, a non-unicode character in a flow cause mitmweb to crash.
2015-07-13 18:19:33 -04:00
isra17
471e196e08 Add ~src ~dst REGEX filters
This filter allow to match on the request source and destination address
in the form of `<IP>:<Port>`.

Also fixed the parsing grammar to add a `WordEnd` after each filter
name. That way, `~src` doesn't match `~s` instead and keep the behavior
consistent with `~hq` != `~h`.
2015-07-13 18:11:14 -04:00
jlhonora
1ebdda7902 Match g/G shortcuts to less, fix #631 2015-07-08 12:10:03 -03:00
Maximilian Hils
5a2b121f50 fix #648 2015-07-04 19:11:02 +02:00
Maximilian Hils
9960565359 fix #653 2015-07-04 15:46:45 +02:00
Terry Long
4c50c36345 Prevent unecessary upstream server connects
Selectively connect to upstream server based on no_upstream_cert
option. When no_upstream_cert is used during server replay, prevent
connecting to the upstream server unless absolutely necessary.
2015-07-03 07:55:37 -07:00
Maximilian Hils
4c831992aa fix #593, fix #656, coverage++ 2015-07-03 02:47:12 +02:00
Kyle Morton
f0ad1f334c Enabling upstream server verification. Added flags --verify_upstream_cert,
--upstream-trusted-cadir, and --upstream-trusted-ca.
2015-06-29 11:00:20 -07:00
Maximilian Hils
b369962cbe remove certforward feature
The certforward feature was implemented to support #gotofail,
which only works on unpatched iOS devices. Given that many apps don't
support iOS 7 anymore, jailbreak+ssl killswitch is usually the better option.
By removing certforward, we can make netlib a pure python module again,
which significantly simplifies distribution.
2015-06-26 13:27:40 +02:00
Aldo Cortesi
876252eba8 Remove stray print & massage whitespace a bit 2015-06-25 10:27:22 +12:00
Thomas Kriechbaumer
14e49f4fc7 unify SSL version/method handling 2015-06-23 12:05:34 +02:00
iroiro123
fbb23b5c9f changed error handling (ssl spoof mode) 2015-06-23 01:49:22 +09:00
iroiro123
fd90367329 SSL Spoof mode 2015-06-21 00:51:56 +09:00
iroiro123
378aa78324 Spoof mode 2015-06-20 21:43:50 +09:00
iroiro123
59ec291b6c HTTP Transparent Proxy 2015-06-18 23:53:27 +09:00
Aldo Cortesi
7890450b0c Handle invalid IDNA encoding in hostnames
Fixes #622
2015-06-12 16:00:16 +12:00
Jake Drahos
946030367f Added unmark all functionality
- 'U' to unmark all marked flows
2015-06-11 20:31:54 -05:00
Jake Drahos
2a6698bf5a Moved marking from flow to console
No longer taints the flow primitive
2015-06-11 20:27:33 -05:00
Jake Drahos
dd1e401e01 Changed mark symbol
Smaller symbol now, still just as easy to see while scrolling
2015-06-11 16:33:46 -05:00
Jake Drahos
486177edc7 Added functionality to write marked flows to file
w (write) -> m (marked)
2015-06-11 16:25:28 -05:00
Jake Drahos
122ee88021 Unmark duplicated flows
If a marked flow is duplicated, the duplicate will now be unmarked
2015-06-11 12:52:15 -05:00
Jake Drahos
13e71eba10 Changed symbols and colors
Added a better symbol for the mark, and changed the color to red. This helps it
stand out more easily.
2015-06-11 12:02:15 -05:00
Jake Drahos
a34eeb9a28 Fixed console rendering bug
Clearing all flows now works properly
2015-06-11 11:49:23 -05:00
Jake Drahos
e53a2426c1 Marked flows not deleted on clear all
Marked flows survive a clear all unless all current flows are marked.

Bug: They don't show up until another flow is added
2015-06-11 11:40:03 -05:00
Jake Drahos
8b998cfbea Implemented basic marking of flows
- Press m to toggle flow mark
- Flow mark is set in libmproxy/console/common.py. Currently set to "==="
2015-06-11 10:27:48 -05:00
Maximilian Hils
d389b9c59d fix #615 2015-06-08 19:01:32 +02:00
Maximilian Hils
083b404bba fix #618 2015-06-08 18:26:02 +02:00
Marcelo Glezer
1befa9477c fix #607 fix message 2015-06-05 15:33:36 -03:00
Marcelo Glezer
17b34de28d fix #607 decode data before sending it to pyperclip 2015-06-05 15:19:57 -03:00
Aldo Cortesi
783e904b92 Merge pull request #603 from Kriechi/remove-contrib
replace contrib libraries with pypi dependencies
2015-06-04 13:30:18 +12:00
Aldo Cortesi
249bb4a2ac Bump version 2015-06-04 11:15:29 +12:00
Thomas Kriechbaumer
fbb2633dd3 replace contrib libraries with pypi dependencies 2015-06-02 10:00:34 +02:00
Aldo Cortesi
0b8cddddf5 console: click-enable tabs 2015-06-02 15:25:58 +12:00
Aldo Cortesi
57a61ae8fd console: convert add_event to a signal. 2015-06-02 13:45:19 +12:00
Aldo Cortesi
776f0a9669 Minor typo in script error. 2015-06-02 11:58:16 +12:00
Aldo Cortesi
1076c25e5b console: click in flow list to view flow 2015-06-02 11:27:26 +12:00
Aldo Cortesi
b5bb4106fd console: mouse scrollwheel throughout. 2015-06-02 11:13:12 +12:00
Aldo Cortesi
62330e4b0f Enable mouse interaction, add a hint for selecting text.
Fixes #597
2015-06-02 11:05:45 +12:00
Marcelo Glezer
14bce0dd12 fixes #604 catch method not found for handling pyperclip not found 2015-06-01 12:06:46 -03:00
Aldo Cortesi
7439e244a6 Merge branch 'master' of ssh.github.com:mitmproxy/mitmproxy 2015-06-01 12:32:03 +12:00
Jim Shaver
07cc7f6f50 reverting change in libmproxy/contrib/pyparsing.py 2015-05-31 08:09:03 -04:00
Jim Shaver
bd019a7124 revert changes in ncontrib 2015-05-31 07:59:00 -04:00
Jim Shaver
b51363b3ca Merge remote-tracking branch 'upstream/master' into print-bracket-fix
Conflicts:
	examples/har_extractor.py
	examples/nonblocking.py
	examples/read_dumpfile
	libmproxy/web/app.py
2015-05-31 01:21:44 -04:00
Aldo Cortesi
6c3233ce0c Appease the code style gods. 2015-05-31 13:16:03 +12:00
Aldo Cortesi
13ee54da86 Remove dependency on PyOpenSSL, move version check into netlib.
There are a few more functions in the code that accesses PyOpenSSL
directly that should probably also be moved to netlib. Later.
2015-05-31 13:13:12 +12:00
Jim Shaver
4fe2c069cc Fixed print function to be inline with python 3 2015-05-29 23:17:48 -04:00
Aldo Cortesi
a05a70d816 Add coding style check, reformat. 2015-05-30 12:03:28 +12:00
Aldo Cortesi
1a106f4080 Ensure that flow/view state is consistent.
Fixes #595 and #592
2015-05-29 11:36:56 +12:00
Aldo Cortesi
99c691def5 Post-release version bump, and checklist addition. 2015-05-26 10:44:52 +12:00
Aldo Cortesi
14341ffa19 Merge pull request #584 from dlethin/feature/http_1_0
Make sure proxy returns the httpversion specified in the request
2015-05-18 11:03:05 +12:00
Aldo Cortesi
33169d610c Release pep: bump version, changelog, contributors 2015-05-18 10:44:09 +12:00
Doug Lethin
cdff79fd4c Make sure proxy returns the httpversion specified in the request rather
than hardcoding to 1.1.
2015-05-15 22:20:09 -04:00
Nick Raptis
202ede3c54 Accurately timestamp start of request
When building a request from a stream, try to get an accurate
start timestamp from the Reader. This was already in the code
and also used when building response objects, but was ommited
in commit ddf458b330

Without his logic and when the client is reusing a connection
to send requests, the timestamp_start of subsequent requests
is early and equal to when the connection started read blocking
2015-05-08 10:58:05 +03:00
Aldo Cortesi
e1d3ebb446 Merge branch 'master' of ssh.github.com:mitmproxy/mitmproxy 2015-05-02 17:20:09 +12:00
Aldo Cortesi
50b7bfaadc Adapt to new Pathoc API - silence output to stdout. 2015-05-02 16:46:15 +12:00
Maximilian Hils
bb965b8e34 web: try harder to fix editor 2015-05-01 20:45:31 +02:00
Maximilian Hils
c780fc7bdf Merge branch 'master' of github.com:mitmproxy/mitmproxy 2015-05-01 17:25:00 +02:00
Maximilian Hils
3f5ca10c39 mitmweb: add editor 2015-05-01 17:24:44 +02:00
Aldo Cortesi
1c26516b18 pretty_size now lives in netlib.utils 2015-04-30 12:18:01 +12:00
Maximilian Hils
90dff4a8a1 Merge pull request #569 from mike-pt/master
Match all freebsd versions and not just freebsd10
2015-04-27 04:41:51 +02:00
Mike C
b942f98f37 + Use sys.platform.startswith("freebsd") instead of matching just freebsd10.
+ This means support for any freebsd version (note that I only tested 11-CURRENT and 9)
2015-04-27 02:31:58 +01:00
Maximilian Hils
1742017752 make code more pythonic 2015-04-26 19:25:59 +02:00
Maximilian Hils
417190daa7 Merge remote-tracking branch 'tunz/crash1' 2015-04-26 18:44:40 +02:00
Maximilian Hils
b5690b8103 remove superfluous set_focus 2015-04-26 18:43:27 +02:00
Maximilian Hils
f96e4957b1 Merge branch 'master' of github.com:mitmproxy/mitmproxy 2015-04-26 18:41:40 +02:00
Maximilian Hils
79e587fe04 fix #568 2015-04-26 18:41:27 +02:00
Choongwoo Han
ad33d0925f Fix crash when save to clipboard before loading 2015-04-23 15:40:57 +09:00
Choongwoo Han
f4f57e62e5 Check overwrite, when save to file instaed of clipboard 2015-04-23 15:02:57 +09:00
Aldo Cortesi
37d731aacd Missed some un-needed variable declarations 2015-04-21 11:42:27 +12:00
Aldo Cortesi
ddf458b330 HTTP request reading moves to netlib 2015-04-21 11:05:37 +12:00
Aldo Cortesi
ff654730e8 console: make g/G shortcuts work in more contexts 2015-04-17 13:30:49 +12:00
Aldo Cortesi
d20069fcd2 console: more consistent view stack management 2015-04-17 13:06:45 +12:00
Aldo Cortesi
e963a9da48 console: suport unary attributes
Attributes with no value are treated as unary, e.g. "Secure" rather than
"Secure=". If you really want to have an empty attribute value you can
edit the header directly.

Behind the scenes, restructure GridEditor to know about data conversion
in and out of the editor.
2015-04-17 12:54:29 +12:00
Aldo Cortesi
7abaf3c362 console: refactor grideditor, fix a crash on tab for subeditors 2015-04-16 22:01:54 +12:00
Aldo Cortesi
cb880cc65a Adjust header key color in solarized palettes 2015-04-16 12:16:07 +12:00
Aldo Cortesi
52716e3439 console: first pass of a Set-cookie editor for responses 2015-04-16 10:57:12 +12:00
Aldo Cortesi
850a50262b console: add a -l flag to set the limit on startup 2015-04-15 12:56:43 +12:00
Aldo Cortesi
5ff4303124 Adjust for ODict interface change 2015-04-15 10:29:57 +12:00
Aldo Cortesi
f33b483110 Editor for request cookies 2015-04-15 09:43:15 +12:00
Aldo Cortesi
c335c2b533 Add set_cookies method to HTTPResponse 2015-04-14 16:23:51 +12:00
Aldo Cortesi
ab7e2857cc New get_cookies for HttpResponse 2015-04-14 15:14:36 +12:00
Aldo Cortesi
e17eacd8d7 New get_cookie and set_cookie implementations for HTTPRequest 2015-04-14 13:45:38 +12:00
Aldo Cortesi
937a358aa9 Merge branch 'master' of ssh.github.com:mitmproxy/mitmproxy 2015-04-14 12:01:37 +12:00
Aldo Cortesi
bea0bd236a Housekeeping and cleanups
- No output to stdout on load in examples - they muck up the test suite.
- Use the odict module directly, rather than aliasing it. The small convenience
this gives to scripters is not worth it.
- Move the cookie tests from the flow test module to the protocol_http test
module.
2015-04-14 11:58:10 +12:00
Maximilian Hils
ab2b98b988 fix pretty_host if no host is present 2015-04-10 19:35:42 +02:00
Maximilian Hils
6852eb9d0a fix #553 2015-04-10 14:59:38 +02:00
Maximilian Hils
1a666bb5db Merge pull request #547 from macmantrl/blank_values
Keep blank query values for flow hashing
2015-04-08 01:22:23 +02:00
Maximilian Hils
e7b154c823 Merge branch 'master' of github.com:mitmproxy/mitmproxy 2015-04-08 00:20:37 +02:00
Maximilian Hils
2242c671fd allow passing custom argv to libmproxy.main 2015-04-08 00:20:32 +02:00
Maximilian Hils
b961f9e599 Merge pull request #543 from legendtang/master
web: fix delete flows
2015-04-08 00:05:04 +02:00
Aldo Cortesi
8e2e83a3c6 console: add g/G shortcuts throughout
g: go to end
G: go to beginning
2015-04-07 16:13:42 +12:00
Aldo Cortesi
e76467e977 Refactor flow list state management
- Use signal mechanism for state synchronisation
- Move "Copy to clipboard" shortcut to "P"
2015-04-07 15:59:38 +12:00
Aldo Cortesi
c7b1234e13 consone: don't enter Options if it's already open 2015-04-07 12:26:56 +12:00
Aldo Cortesi
6bab6f2ef1 console: tune palettes a bit 2015-04-07 11:36:42 +12:00
Aldo Cortesi
94cd704f73 console: toggle palette transparency from options, indicate selected palette 2015-04-07 10:11:25 +12:00
Aldo Cortesi
f6a3bd15b3 console: All palettes now explicitly set a background colour
There's a new option --palette-transparent to turn this off if you want
to use your own terminal background.
2015-04-07 10:01:18 +12:00
Aldo Cortesi
1cb1ee411b console: palette picker for the options screen 2015-04-07 08:42:40 +12:00
Aldo Cortesi
538f215458 console: factor out selection widget 2015-04-06 18:28:51 +12:00
Aldo Cortesi
0d6de19b07 console: sticky cookies and auth to options screen 2015-04-06 17:30:12 +12:00
Aldo Cortesi
c4e0f9d8d7 console: TCP proxy patterns to new options screen 2015-04-06 17:19:52 +12:00
Aldo Cortesi
ec7572697a console: default display mode to new options screen 2015-04-06 17:14:32 +12:00
Aldo Cortesi
acb6b5667c console: scripts to new options screen 2015-04-06 17:11:02 +12:00
Aldo Cortesi
488adcb79e console: replacement patterns to new option screen 2015-04-06 17:05:52 +12:00
Aldo Cortesi
15246c3403 console: ignore patterns to new options screen. 2015-04-06 17:02:51 +12:00
Aldo Cortesi
9e39999706 Add headings to options, start migrating more options into options screen
SetHeaders first...
2015-04-06 16:47:58 +12:00
Legend Tang
20d3d7e1b8 Merge https://github.com/mitmproxy/mitmproxy 2015-04-06 11:28:20 +08:00
Aldo Cortesi
3704411466 Minimal help context for options 2015-04-03 20:04:19 +13:00
Aldo Cortesi
aadaa66d0b console: tune and re-enable palettes 2015-04-03 19:44:09 +13:00