Commit Graph

529 Commits

Author SHA1 Message Date
Shadab Zafar
d361177753 h2: move header parsing to netlib 2016-07-03 18:03:34 +05:30
Shadab Zafar
f623b3d99b py3++: test_flow_export 2016-07-02 21:15:20 +05:30
Shadab Zafar
17b727321f Replace map + lambda with list comprehensions 2016-07-02 21:15:20 +05:30
Shadab Zafar
393c1d74af fix lint issues 2016-07-02 18:56:22 +05:30
Maximilian Hils
d9f797e7e6 make the linter happy 2016-07-02 02:11:00 -07:00
Maximilian Hils
6032c4f235 message.content -> .raw_content, implement .text
This PR improves our handling of HTTP message body encodings:

- The unaltered message body is now accessible as `.raw_content`
- The "content-encoding"-decoded content (i.e. gzip removed) content
  is not `.content`, as this is what we want in 99% of the cases.
- `.text` now provides the "content-encoding"-decoded and then
  "content-type charset"-decoded message body.
- The decoded values for `.content` and `.text` are cached,
  so that repeated access and `x.text = x.text` is cheap.
- The `decoded()` decorator is now deprecated, as we can now just use
  `.content`. Similarly `HTTPMessage.get_decoded_content()` is
  deprecated.
2016-07-02 01:51:47 -07:00
Maximilian Hils
fa72462798 fix tcp message handling 2016-07-01 17:17:16 -07:00
Maximilian Hils
536c7acd13 py3++ 2016-07-01 14:10:48 -07:00
Shadab Zafar
268e42e55e py3++: test/mitmproxy/test_server 2016-07-01 10:23:04 -07:00
Maximilian Hils
8366c87c57 update to latest typing release 2016-06-29 03:03:23 -07:00
Maximilian Hils
025efc030a Merge pull request #1301 from mitmproxy/simplify-tox
Simplify tox
2016-06-29 02:04:38 -07:00
Maximilian Hils
f7d4319c5a minor fixes 2016-06-28 23:48:19 -07:00
Maximilian Hils
b7430c0775 Merge pull request #1296 from gzzhanghao/test
[web] add tests for ducks/utils
2016-06-28 23:16:18 -07:00
Maximilian Hils
a57ea27bd4 watchdog: always use PollingObserver 2016-06-28 22:09:12 -07:00
Maximilian Hils
3b53d3e393 fix mysterious tox issue 2016-06-28 19:26:08 -07:00
Jason
37c2b47c26 [web] add tests for utils 2016-06-27 18:42:20 +08:00
Maximilian Hils
3eb2d04aac minor fixes 2016-06-26 01:45:45 -07:00
Jason
db415a26bc [web] log only in dev 2016-06-25 10:50:15 +08:00
Jason
a782571244 [web] build with envify 2016-06-25 10:43:30 +08:00
Jason
59dc5e4687 [web] update build 2016-06-25 10:09:21 +08:00
Jason
e400e9f513 [web] ErrorView 2016-06-25 09:52:49 +08:00
Jason
2b7923b4f4 [web] fix actions 2016-06-25 02:18:52 +08:00
Jason
571c817f21 [web] add msgQueue 2016-06-25 00:36:39 +08:00
Jason
5a1677c387 Merge branch 'master' of github.com:mitmproxy/mitmproxy into list
Conflicts:
	mitmproxy/web/static/app.js
	web/src/js/app.jsx
	web/src/js/connection.js
	web/src/js/ducks/websocket.js
2016-06-24 21:04:39 +08:00
Maximilian Hils
21c3480429 Merge remote-tracking branch 'jason/websocket' 2016-06-23 23:38:19 -07:00
Jason
f392b8e34c [web] bug fix 2016-06-24 01:20:42 +08:00
Jason
c40ff0277b [web] fix bugs 2016-06-24 01:01:34 +08:00
Jason
397554b7c7 [web] update build 2016-06-23 01:30:34 +08:00
Shadab Zafar
d655d8f1f3 py3++: mitmproxy.platform.pf 2016-06-22 13:38:00 +05:30
Shadab Zafar
119021a9c3 py3++: mitmproxy.proxy 2016-06-22 13:37:16 +05:30
Jason
57af1d00dd [web] set connected to false when DISCONNECTED 2016-06-22 11:29:21 +08:00
Jason
26aed8f56f [web] updateSettings -> handleWsMsg 2016-06-22 11:24:22 +08:00
Shadab Zafar
0a53550955 mitmproxy.tnetstring -> mitmproxy.contrib.tnetstring 2016-06-21 23:53:59 +05:30
Shadab Zafar
b5d280cab1 Py3: tnetstring
Copied the initial code from
[tnetstring3](https://raw.githubusercontent.com/carlopires/tnetstring3/master/tnetstring/__init__.py)

Then made some changes, like using the isdigit function, removing
superfluous ord() etc. to make all our tests pass.
2016-06-21 23:37:28 +05:30
Jason
9a7415a58d [web] store ws object in symbol property 2016-06-22 00:51:45 +08:00
Jason
0cab9ef1fa Merge remote-tracking branch 'origin/master' into websocket
Conflicts:
	mitmproxy/web/static/app.js
	web/src/js/components/ProxyApp.jsx
2016-06-22 00:49:21 +08:00
Maximilian Hils
7ef0e37c74 Merge pull request #1273 from Kriechi/fix-default-arg
fix default arg
2016-06-20 14:52:16 -07:00
Shadab Zafar
5b95803be0 py3++ 2016-06-20 21:57:18 +02:00
Shadab Zafar
efeade134a py3++ 2016-06-20 21:39:39 +02:00
Maximilian Hils
165deb08fd Merge remote-tracking branch 'jason/settings' 2016-06-19 18:27:12 -07:00
Thomas Kriechbaumer
6ad0e3e66e fix default arg 2016-06-19 12:05:10 +02:00
Thomas Kriechbaumer
7bde38dc4a py3++ 2016-06-19 12:00:37 +02:00
Jason
d01e49c6d5 [web] replace SetttingsActions.update with settings.updateSettings 2016-06-19 02:53:15 +08:00
Doug Freed
1f915b7f24 mitmproxy/models: missed comma in __all__
Spotted by Landscape (thanks @Kriechi for pointing out the site).  Add a
comma to the last item too, to prevent this from happening in the future
and reduce messing up blame later.
2016-06-18 14:50:06 -04:00
Jason
9cb5b0af9d Merge branch 'master' of github.com:mitmproxy/mitmproxy into websocket
Conflicts:
	mitmproxy/web/static/app.js
	web/src/js/components/ProxyApp.jsx
	web/src/js/connection.js
2016-06-18 14:18:24 +08:00
Jason
6c0511b06f [web] add app ducks 2016-06-18 14:11:47 +08:00
Maximilian Hils
ce53799c62 simplify settings reducer 2016-06-17 21:03:19 -07:00
Jason
965b27b529 [web] simplify settings 2016-06-17 22:12:32 +08:00
Jason
c82d27b2a2 Merge branch 'master' into settings
Conflicts:
	mitmproxy/web/static/app.css
	mitmproxy/web/static/app.js
	web/src/js/components/ProxyApp.jsx
2016-06-17 21:58:24 +08:00
Jason
e4addd60f1 Merge branch 'master' into websocket
Conflicts:
	mitmproxy/web/static/app.css
	mitmproxy/web/static/app.js
	web/src/js/components/ProxyApp.jsx
2016-06-17 21:54:31 +08:00
Jason
034287edcf [web] clean up 2016-06-17 07:57:06 +08:00
Jason
16a28eca25 [web] websocket 2016-06-17 07:26:29 +08:00
Jason
7a78970d10 [web] common 2016-06-17 07:07:38 +08:00
Jason
6ad2f13341 [web] fix settings ducks 2016-06-17 06:01:29 +08:00
Jason
cedac98b70 [web] promot reduce 2016-06-17 05:08:05 +08:00
Jason
7fb0836bfe [web] remove store 2016-06-17 05:08:05 +08:00
Jason
99b2b8633d [web] settings store 2016-06-17 05:08:05 +08:00
Jason
f5c597a9e3 [web] Editor & Prompt 2016-06-17 04:38:35 +08:00
Jason
e5bf1e930a [web] FlowView and ContentView 2016-06-17 04:37:57 +08:00
Maximilian Hils
c7a891b652 Merge branch 'add_flow_to_options' 2016-06-15 13:40:03 -07:00
Clemens
48e372bf98 rename view -> ui, make code more readable 2016-06-15 08:54:55 +02:00
Clemens
00b4abfe83 add previous state to dispatch function 2016-06-15 00:19:33 +02:00
Clemens
003096b4cc code review 2016-06-14 22:17:34 +02:00
Clemens
bf75e14573 added flowMenu 2016-06-14 11:20:48 +02:00
Aldo Cortesi
c2b5a13e3f Don't fail when start() is not defined 2016-06-14 14:39:07 +12:00
Maximilian Hils
804efe9d38 fix contextmanager, remove args argument to start() 2016-06-13 18:08:57 -07:00
Aldo Cortesi
e7b61995ad A tiny wrapper to make script invocations nicer 2016-06-14 12:34:22 +12:00
Thomas Kriechbaumer
a5aa16e03b fix #1240 2016-06-12 11:51:49 +02:00
Aldo Cortesi
728d5caa22 Fold mitmproxy.version and pathod.version into netlib.version 2016-06-12 10:29:12 +12:00
Aldo Cortesi
7b86560ded debug: On SIGUSR2, we dump tracebacks for all threads to screen 2016-06-11 23:07:42 +12:00
Aldo Cortesi
09edbd9492 Improve debugging of thread and other leaks
- Add basethread.BaseThread that all threads outside of test suites should use
- Add a signal handler to mitmproxy, mitmdump and mitmweb that dumps resource
information to screen when SIGUSR1 is received.
- Improve thread naming throughout to make thread dumps understandable
2016-06-11 19:52:24 +12:00
Aldo Cortesi
e367b88195 Add a --sysinfo flag to all daemons
This dumps all the platform information and mitmproxy version data we'd
normally need to troubleshoot an issue.
2016-06-11 16:40:21 +12:00
Jason
c33df55919 [web] update build 2016-06-09 20:39:51 +08:00
Jason
81a0c45c89 [web] header.js -> Header.js 2016-06-09 20:35:03 +08:00
Jason
6c95635cb8 [web] eventlog.js -> EventLog.jsx 2016-06-09 18:40:59 +08:00
Jason
1baefcdc99 [web] footer.js -> Footer.jsx 2016-06-09 18:13:18 +08:00
Jason
e24bf8d73f [web] fix shortcut for header fields 2016-06-09 18:03:40 +08:00
Jason
f306cfa8b6 [web] separate flowtable to multiple files 2016-06-09 17:52:30 +08:00
Jason
6407765285 [web] update build 2016-06-09 14:22:23 +08:00
Jason
8e538c7630 [web] fix updateLocation 2016-06-09 14:20:14 +08:00
Aldo Cortesi
c421c41307 Remove odict
- Adds default implementations for _kconv and _reduce_values to MultiDict.
Without these, operations fail in really, really non-obvious ways.
- Replace the remaining few instances of ODict

Fixes #1159
2016-06-09 13:28:43 +12:00
Maximilian Hils
90cb84b536 Merge pull request #1220 from mitmproxy/add_sorting_with_redux
Add sorting with redux
2016-06-08 13:37:09 -07:00
redfast00
16d32b2ce5
fix after cleanup of netlib.utils in mitmproxy.console.statusbar 2016-06-08 13:07:19 +02:00
Clemens
ad84d650de sorting add pr review changes 2016-06-08 10:29:33 +02:00
Aldo Cortesi
a5cb241c7c If a message has been acked, all other processors are skipped
This applies the constraint, but does to clumsily. When we've unified modules
and processors it will be much nicer. We also make some exceptions for the
master processors that we may want to re-evaluate down the track.
2016-06-08 12:58:58 +12:00
Aldo Cortesi
b3bf754e53 Simplify script concurrency helpers
We now have take() to prevent double-replies.
2016-06-08 11:21:38 +12:00
Aldo Cortesi
a388ddfd78 A new interface for reply
Reply is now explicit - it's no longer a callable itself. Instead, we have:

    reply.kill()            - kill the flow
    reply.ack()             - ack, but don't send anything
    reply.send(message)     - send a response

This is part of an incremental move to detach reply from our flow objects,
and unify the script and handler interfaces.
2016-06-08 10:44:20 +12:00
Aldo Cortesi
982077ec31 Add reply.ack and reply.kill 2016-06-08 10:14:34 +12:00
Clemens
dbbda03020 sorting done, ready to review 2016-06-07 11:22:52 +02:00
Clemens
fa3ad2382b sorting done, ready to review 2016-06-07 11:07:25 +02:00
Clemens
1aac283e23 moved redux state injection closer to child 2016-06-07 11:07:25 +02:00
Clemens
ff2129773f start with sort, continue tomorrow 2016-06-07 11:07:25 +02:00
Clemens
ac2d8549e5 moved sort state to redux, missing apply sort 2016-06-07 11:07:25 +02:00
Maximilian Hils
ba466dbb35 simplify pretty_json, improve test coverage 2016-06-06 23:08:58 -07:00
Maximilian Hils
71b6e369f8 change content views: escape in raw mode, show utf8 in json 2016-06-06 22:42:11 -07:00
Aldo Cortesi
1ffc273c94 Utils cleanups
- Move more stuff that belongs in netlib.human
- Move some stuff to near the only use
- Zap mitmproxy.utils.timestamp(). I see the rationale, but we used it
interchangeably with time.time() throughout the project. Since time.time()
dominates in the codebase and timestamp() is such low utility, away it goes.
2016-06-07 17:12:52 +12:00
Maximilian Hils
38d2e6432c minor fixes 2016-06-06 19:10:19 -07:00
Aldo Cortesi
e9fb35b817 Fix lint 2016-06-06 15:02:16 +12:00
Maximilian Hils
d878122cf9 web: fix list update propagation 2016-06-05 18:43:17 -07:00
Maximilian Hils
b6ebde3b1d Merge branch 'such-redux' 2016-06-05 18:23:46 -07:00
Maximilian Hils
e6ef149a03 web: modernize flow table columns 2016-06-05 18:23:06 -07:00
Aldo Cortesi
2b19a33738 Merge pull request #1212 from mitmproxy/such-redux
web: completely move flow state to redux
2016-06-06 08:58:50 +12:00
Maximilian Hils
d53a2de0ba web: completely move flow state to redux 2016-06-04 18:53:41 -07:00
Thomas Kriechbaumer
1c4e1e049c http2: remove custom PUSH_PROMISE header 2016-06-04 11:20:10 +02:00
Thomas Kriechbaumer
ac70eaed66 http2: improve GoAway handling 2016-06-04 11:20:10 +02:00
Thomas Kriechbaumer
7f4ac6f27b http2: respect MAX_CONCURRENT_STREAMS by hold-off 2016-06-04 11:20:10 +02:00
Maximilian Hils
e880f532ad web interleave old store and redux flow state 2016-06-04 00:58:13 -07:00
Maximilian Hils
c0d08be7a6 Merge branch 'master' of https://github.com/mitmproxy/mitmproxy 2016-06-04 00:41:35 -07:00
Maximilian Hils
b02d66491c web: implement update and remove for list and view 2016-06-04 00:37:10 -07:00
Aldo Cortesi
d5532319c1 Basic ConsoleMaster tests, based on mastertester 2016-06-04 19:07:27 +12:00
Aldo Cortesi
af63a3e064 console/__init__.py -> console/master.py 2016-06-04 16:06:44 +12:00
Aldo Cortesi
f7f9cab5dc Rebase on master 2016-06-04 14:37:07 +12:00
Maximilian Hils
1b1ea98f08 web: add flows to redux state 2016-06-03 18:53:30 -07:00
Maximilian Hils
1b327f34c3 web: add redux list views 2016-06-03 18:38:25 -07:00
Maximilian Hils
d6fcd7e06d web: implement redux store fetching 2016-06-03 18:38:25 -07:00
Thomas Kriechbaumer
46d9cdd70a fix style 2016-06-03 18:57:46 +02:00
Maximilian Hils
e31aa39fc2 Merge branch 'redux-ducks' 2016-06-02 23:45:36 -07:00
Maximilian Hils
65fde7f554 Merge pull request #1190 from mitmproxy/file_menu_save_open
File menu save open
2016-06-02 23:42:35 -07:00
Maximilian Hils
5321f15def web: fix eventlog height registration 2016-06-02 23:41:32 -07:00
Maximilian Hils
7afac747a8 web: reduxify event log store 2016-06-02 23:40:30 -07:00
Maximilian Hils
7c63aa3708 web: reduxify WebSocket 2016-06-02 21:12:59 -07:00
Aldo Cortesi
1ab64da682 Merge pull request #1167 from xntrik/mark_filter
implement a toggle for viewing marked flows only in console
2016-06-03 15:05:18 +12:00
Aldo Cortesi
08e4cd2a40 Merge remote-tracking branch 'upstream/master' 2016-06-03 14:55:35 +12:00
Aldo Cortesi
25671b91bd Fix flow handling for mitmdump
Fixes #1188
2016-06-03 14:53:51 +12:00
Maximilian Hils
698d38b28e web: 🐣 2016-06-02 17:46:18 -07:00
Clemens
5e7fa349a9 final review done, nothing to change -> ready to merge 2016-06-02 21:01:22 +02:00
Maximilian Hils
9c6da08d00 fix dumpfile upload 2016-06-02 11:41:26 -07:00
Maximilian Hils
73e494770f web: add fetchApi convenience method 2016-06-02 10:34:16 -07:00
Clemens
89fc438e32 change some pr issuses 2016-06-02 10:31:24 -07:00
Clemens
a219d33430 basic file up and download working 2016-06-02 10:30:53 -07:00
Clemens
dfc033ab5f added basic file up_download functionality (not finish yet) 2016-06-02 10:30:43 -07:00
Maximilian Hils
d84610a41d web: use object spread operator
While the spread operator is not part of ES6, React also uses it for JSX.
It makes sense for us to keep it in other parts of the codebase as well.
2016-06-02 10:24:33 -07:00
Maximilian Hils
7a388560d7 use object spread operator
While the spread operator is not part of ES6, React also uses it for JSX.
It makes sense for us to keep it in other parts of the codebase as well.
2016-06-02 09:51:39 -07:00
Maximilian Hils
9e3591e3c2 add basic redux architecture, convert eventlog to redux 2016-06-02 00:04:10 -07:00
Aldo Cortesi
09da1febbd Shift a bunch more string-related functions to strutils 2016-06-02 13:04:19 +12:00
Aldo Cortesi
cccdc98426 Utils reorganisation: add netlib.strutils
Extract a number of string and format-related functions to netlib.strutils.
2016-06-02 12:31:41 +12:00
Aldo Cortesi
eaa3b308f7 Fix non-deterministic test failures in export
We had various places in the code where we relied on incidental order of dict
keys. Add a helper to multidict, and fix.
2016-06-02 11:37:18 +12:00
Aldo Cortesi
1a08048a8d Un-break mitmproxy console 2016-06-01 16:51:17 +12:00
Aldo Cortesi
137ee28bd6 Fix test suite 2016-06-01 15:10:54 +12:00
Aldo Cortesi
d60fdebc47 Fix lint harder 2016-06-01 14:50:06 +12:00
Aldo Cortesi
efbdd8d266 Merge branch 'module-imports' of github.com:mitmproxy/mitmproxy into module-imports 2016-06-01 14:43:24 +12:00
Aldo Cortesi
afcfa783ea Satisfy lint 2016-06-01 14:43:08 +12:00
Maximilian Hils
1ca45b1e64 fix imports 2016-05-31 19:35:10 -07:00
Maximilian Hils
3aa8d42b05 fix PIL import 2016-05-31 19:25:47 -07:00
Maximilian Hils
0a2e7a97cd use module-level imports only 2016-05-31 19:06:57 -07:00
Maximilian Hils
30fff1fb32 be more strict about module namespaces 2016-05-31 14:05:57 -07:00
Aldo Cortesi
42e91fcfe1 http2.frame -> http2.framereader 2016-05-31 20:32:28 +12:00
Aldo Cortesi
b2f63458fc Move human-friendly format functions to netlib.human, remove redundant implementations 2016-05-31 19:32:08 +12:00
Aldo Cortesi
ec34cae618 utils.multipartdecode -> http.multipart.decode
also

utils.parse_content_type -> http.headers.parse_content_type
2016-05-31 19:07:55 +12:00
Aldo Cortesi
6dda2b2ee5 Module is part of the name - url.decode, not url.urldecode
A pattern we need to use far more often in the codebase
2016-05-31 18:46:19 +12:00