Grasscutters/mitmproxy
3
0
Fork 0
mirror of https://github.com/Grasscutters/mitmproxy.git synced 2024-11-27 02:24:18 +00:00
Code Issues Projects Releases Wiki Activity
35f4a1c424
mitmproxy/test/test_utils.py

256 lines
7.2 KiB
Python
Raw Normal View History

Removed imports left unused after Py{OpenSSL,ASN1} Commits 533f61f67aab38f5bce882ad0dc03b7b5f292956 and 8b841bc9e370370716b473f26e001c65e2eee2af left some imports unused while swithing to PyOpenSSL and PyASN1 -- this commit removes these imports.
2012-03-01 15:20:34 +00:00
import textwrap, re, json
Initial checkin.
2010-02-16 04:09:07 +00:00
import libpry
from libmproxy import utils
Add a hideous kludge to fix not-yet-valid certificates. - The OpenSSL x509 has no way to explicitly set the notBefore value on certificates. - If two systems have the same configured time, it's possible to return a certificate before the validity start time has arrived. - We "solve" this by waiting for one second when a certificate is first generated before returning the cert. The alternative is to rewrite pretty much all of our certificate generation, a thought too horrible to contemplate.
2011-06-11 03:16:16 +00:00
utils.CERT_SLEEP_TIME = 0
Initial checkin.
2010-02-16 04:09:07 +00:00
Add timestamps to flows. For now, these are only displayed on the connection view screen, with second granularity.
2011-02-03 00:30:47 +00:00
class uformat_timestamp(libpry.AutoTree):
def test_simple(self):
Store timestamps on flow components as a UTC time tuple. Format is: (tm_year,tm_mon,tm_mday,tm_hour,tm_min, tm_sec,tm_wday,tm_yday,tm_isdst)
2011-03-07 00:46:02 +00:00
assert utils.format_timestamp(utils.timestamp())
Add timestamps to flows. For now, these are only displayed on the connection view screen, with second granularity.
2011-02-03 00:30:47 +00:00
Initial checkin.
2010-02-16 04:09:07 +00:00
class uisBin(libpry.AutoTree):
def test_simple(self):
assert not utils.isBin("testing\n\r")
assert utils.isBin("testing\x01")
assert utils.isBin("testing\x0e")
assert utils.isBin("testing\x7f")
Try not to hang when user views large request & response bodies Two different strategies here: - Use a simple heuristic to detect if we're looking at XML data when indent mode is used. On non-XML data we can hang even on small documents. - Only view partial data for large bodies. At the moment the cutoff is 100k. I might finetune this later.
2011-06-27 03:59:17 +00:00
class uisXML(libpry.AutoTree):
def test_simple(self):
assert not utils.isXML("foo")
assert utils.isXML("<foo")
assert utils.isXML(" \n<foo")
Initial checkin.
2010-02-16 04:09:07 +00:00
class uhexdump(libpry.AutoTree):
def test_simple(self):
assert utils.hexdump("one\0"*10)
Expand test coverage.
2012-02-10 02:04:20 +00:00
Unit test++
2011-08-02 04:52:47 +00:00
class udel_all(libpry.AutoTree):
def test_simple(self):
d = dict(a=1, b=2, c=3)
utils.del_all(d, ["a", "x", "b"])
assert d.keys() == ["c"]
Initial checkin.
2010-02-16 04:09:07 +00:00
Expand test coverage.
2012-02-10 02:04:20 +00:00
class uclean_hanging_newline(libpry.AutoTree):
def test_simple(self):
s = "foo\n"
assert utils.clean_hanging_newline(s) == "foo"
assert utils.clean_hanging_newline("foo") == "foo"
Initial checkin.
2010-02-16 04:09:07 +00:00
class upretty_size(libpry.AutoTree):
def test_simple(self):
assert utils.pretty_size(100) == "100B"
assert utils.pretty_size(1024) == "1kB"
Code cleanliness - appease pychecker.
2011-08-03 22:34:34 +00:00
assert utils.pretty_size(1024 + (1024/2.0)) == "1.5kB"
Initial checkin.
2010-02-16 04:09:07 +00:00
assert utils.pretty_size(1024*1024) == "1M"
class uData(libpry.AutoTree):
def test_nonexistent(self):
Cleanliness fixes. - Remove unused code during previous commit. - Code coverage fixes.
2012-03-11 22:25:50 +00:00
assert utils.pkg_data.path("console")
General cleanup. Cut out unused variables and code, generally shut up pychecker as much as is reasonable.
2011-08-02 04:14:33 +00:00
libpry.raises("does not exist", utils.pkg_data.path, "nonexistent")
Initial checkin.
2010-02-16 04:09:07 +00:00
Rip out BeautifulSoup, and use a custom XML-ish prettyprinter.
2011-02-06 01:17:30 +00:00
class upretty_xmlish(libpry.AutoTree):
def test_tagre(self):
def f(s):
return re.search(utils.TAG, s, re.VERBOSE|re.MULTILINE)
assert f(r"<body>")
assert f(r"<body/>")
assert f(r"< body/>")
assert f(r"< body/ >")
assert f(r"< body / >")
assert f(r"<foo a=b>")
assert f(r"<foo a='b'>")
assert f(r"<foo a='b\"'>")
assert f(r'<a b=(a.b) href="foo">')
assert f('<td width=25%>')
Further fine-tuning for pretty_xmlish.
2011-02-06 03:56:13 +00:00
assert f('<form name="search" action="/search.php" method="get" accept-charset="utf-8" class="search">')
assert f('<img src="gif" width="125" height="16" alt=&quot;&quot; />')
Rip out BeautifulSoup, and use a custom XML-ish prettyprinter.
2011-02-06 01:17:30 +00:00
Basix XML/HTML pretty-printing in flow viewer.
2011-01-27 04:26:01 +00:00
def test_all(self):
Rip out BeautifulSoup, and use a custom XML-ish prettyprinter.
2011-02-06 01:17:30 +00:00
def isbalanced(ret):
# The last tag should have no indent
assert ret[-1].strip() == ret[-1]
s = "<html><br><br></br><p>one</p></html>"
ret = utils.pretty_xmlish(s)
isbalanced(ret)
s = r"""
<body bgcolor=#ffffff text=#000000 link=#0000cc vlink=#551a8b alink=#ff0000 onload="document.f.q.focus();if(document.images)new Image().src='/images/srpr/nav_logo27.png'" ><textarea id=csi style=display:none></textarea></body>
"""
isbalanced(utils.pretty_xmlish(textwrap.dedent(s)))
s = r"""
<a href="http://foo.com" target="">
<img src="http://foo.gif" alt="bar" height="25" width="132">
</a>
"""
isbalanced(utils.pretty_xmlish(textwrap.dedent(s)))
s = r"""
<!DOCTYPE html PUBLIC \"-//W3C//DTD XHTML 1.0 Strict//EN\"
\"http://www.w3.org/TR/xhtml1/DTD/xhtml1-strict.dtd\">
<html></html>
"""
ret = utils.pretty_xmlish(textwrap.dedent(s))
isbalanced(ret)
s = "<html><br/><p>one</p></html>"
ret = utils.pretty_xmlish(s)
assert len(ret) == 6
isbalanced(ret)
s = "gobbledygook"
Further fine-tuning for pretty_xmlish.
2011-02-06 03:56:13 +00:00
assert utils.pretty_xmlish(s) == ["gobbledygook"]
Rip out BeautifulSoup, and use a custom XML-ish prettyprinter.
2011-02-06 01:17:30 +00:00
Basix XML/HTML pretty-printing in flow viewer.
2011-01-27 04:26:01 +00:00
JSON pretty-printing. Also rename the display modes ("pretty" instead of "indent"), and expand the built-in documentation.
2011-06-30 01:27:27 +00:00
class upretty_json(libpry.AutoTree):
def test_one(self):
s = json.dumps({"foo": 1})
assert utils.pretty_json(s)
assert not utils.pretty_json("moo")
Add a pretty-printing mode for urlencoded form data.
2011-07-15 04:16:43 +00:00
class u_urldecode(libpry.AutoTree):
def test_one(self):
s = "one=two&three=four"
assert len(utils.urldecode(s)) == 2
Improve responsiveness of request and response viewing. - Computing the view of a large body is expensive, so we introduce an LRU cache to hold the latest 20 results. - Use ListView more correctly, passing it individual urwid.Text snippets, rather than a single large one. This hugely improves render time.
2011-03-15 00:05:33 +00:00
class uLRUCache(libpry.AutoTree):
def test_one(self):
class Foo:
ran = False
@utils.LRUCache(2)
def one(self, x):
self.ran = True
return x
f = Foo()
assert f.one(1) == 1
assert f.ran
f.ran = False
assert f.one(1) == 1
assert not f.ran
f.ran = False
assert f.one(1) == 1
assert not f.ran
assert f.one(2) == 2
assert f.one(3) == 3
assert f.ran
f.ran = False
assert f.one(1) == 1
assert f.ran
assert len(f._cached_one) == 2
assert len(f._cachelist_one) == 2
Make pretty-printing more robust. Also, since BeautifulSoup is so damn slow, print a statusbar message saying that we're calculating a pretty version of the response. Maybe I should add hangman or something, becuase on a 200k document this can take ages.
2011-01-27 23:07:27 +00:00
Refactor reverse proxying - Retain the specification from the Host header as a Request's description. - Expand upstream proxy specifications to include the scheme. We now say https://hostname:port - Move the "R" revert keybinding to "v" to make room for a reverse proxy binding that matches the command-line flag.
2012-02-18 01:45:22 +00:00
class u_parse_proxy_spec(libpry.AutoTree):
def test_simple(self):
assert not utils.parse_proxy_spec("")
assert utils.parse_proxy_spec("http://foo.com:88") == ("http", "foo.com", 88)
assert utils.parse_proxy_spec("http://foo.com") == ("http", "foo.com", 80)
Unit test import cleanups.
2012-02-19 22:04:07 +00:00
assert not utils.parse_proxy_spec("foo.com")
assert not utils.parse_proxy_spec("http://")
Refactor reverse proxying - Retain the specification from the Host header as a Request's description. - Expand upstream proxy specifications to include the scheme. We now say https://hostname:port - Move the "R" revert keybinding to "v" to make room for a reverse proxy binding that matches the command-line flag.
2012-02-18 01:45:22 +00:00
Move all HTTP objects to flow.py That's Request, Response, ClientConnect, ClientDisconnect, Error, and Headers.
2011-08-03 10:38:23 +00:00
Firm up reverse proxy specification. - Extract proxy spec parsing and unparsing functions. - Add a status indicator in mitmproxy. - Add the "R" keybinding for changing the reverse proxy from within mitmproxy.
2012-02-18 03:27:09 +00:00
class u_unparse_url(libpry.AutoTree):
def test_simple(self):
assert utils.unparse_url("http", "foo.com", 99, "") == "http://foo.com:99"
assert utils.unparse_url("http", "foo.com", 80, "") == "http://foo.com"
assert utils.unparse_url("https", "foo.com", 80, "") == "https://foo.com:80"
assert utils.unparse_url("https", "foo.com", 443, "") == "https://foo.com"
Move all HTTP objects to flow.py That's Request, Response, ClientConnect, ClientDisconnect, Error, and Headers.
2011-08-03 10:38:23 +00:00
class u_parse_url(libpry.AutoTree):
def test_simple(self):
assert not utils.parse_url("")
u = "http://foo.com:8888/test"
s, h, po, pa = utils.parse_url(u)
assert s == "http"
assert h == "foo.com"
assert po == 8888
assert pa == "/test"
s, h, po, pa = utils.parse_url("http://foo/bar")
assert s == "http"
assert h == "foo"
assert po == 80
assert pa == "/bar"
s, h, po, pa = utils.parse_url("http://foo")
assert pa == "/"
s, h, po, pa = utils.parse_url("https://foo")
assert po == 443
Improve robustness against invalid data.
2011-09-04 19:47:47 +00:00
assert not utils.parse_url("https://foo:bar")
assert not utils.parse_url("https://foo:")
Add HTTP body size limit specification to command-line tools.
2011-09-09 03:27:31 +00:00
class u_parse_size(libpry.AutoTree):
def test_simple(self):
Expand test coverage.
2012-02-10 02:04:20 +00:00
assert not utils.parse_size("")
Add HTTP body size limit specification to command-line tools.
2011-09-09 03:27:31 +00:00
assert utils.parse_size("1") == 1
assert utils.parse_size("1k") == 1024
assert utils.parse_size("1m") == 1024**2
assert utils.parse_size("1g") == 1024**3
libpry.raises(ValueError, utils.parse_size, "1f")
libpry.raises(ValueError, utils.parse_size, "ak")
Add a simple parser for content type specifications.
2012-03-19 21:31:07 +00:00
class u_parse_content_type(libpry.AutoTree):
def test_simple(self):
p = utils.parse_content_type
assert p("text/html") == ("text", "html", {})
assert p("text") == None
v = p("text/html; charset=UTF-8")
assert v == ('text', 'html', {'charset': 'UTF-8'})
Add a variant of cleanBin that escapes newlines and tabs. Use this to fix the hex display option.
2012-03-26 22:25:50 +00:00
class u_cleanBin(libpry.AutoTree):
def test_simple(self):
assert utils.cleanBin("one") == "one"
assert utils.cleanBin("\00ne") == ".ne"
assert utils.cleanBin("\nne") == "\nne"
assert utils.cleanBin("\nne", True) == ".ne"
Initial checkin.
2010-02-16 04:09:07 +00:00
tests = [
Add a variant of cleanBin that escapes newlines and tabs. Use this to fix the hex display option.
2012-03-26 22:25:50 +00:00
u_cleanBin(),
Add a simple parser for content type specifications.
2012-03-19 21:31:07 +00:00
u_parse_content_type(),
Add timestamps to flows. For now, these are only displayed on the connection view screen, with second granularity.
2011-02-03 00:30:47 +00:00
uformat_timestamp(),
Initial checkin.
2010-02-16 04:09:07 +00:00
uisBin(),
Try not to hang when user views large request & response bodies Two different strategies here: - Use a simple heuristic to detect if we're looking at XML data when indent mode is used. On non-XML data we can hang even on small documents. - Only view partial data for large bodies. At the moment the cutoff is 100k. I might finetune this later.
2011-06-27 03:59:17 +00:00
uisXML(),
Initial checkin.
2010-02-16 04:09:07 +00:00
uhexdump(),
upretty_size(),
uData(),
Rip out BeautifulSoup, and use a custom XML-ish prettyprinter.
2011-02-06 01:17:30 +00:00
upretty_xmlish(),
JSON pretty-printing. Also rename the display modes ("pretty" instead of "indent"), and expand the built-in documentation.
2011-06-30 01:27:27 +00:00
upretty_json(),
Add a pretty-printing mode for urlencoded form data.
2011-07-15 04:16:43 +00:00
u_urldecode(),
Unit test++
2011-08-02 04:52:47 +00:00
udel_all(),
Improve responsiveness of request and response viewing. - Computing the view of a large body is expensive, so we introduce an LRU cache to hold the latest 20 results. - Use ListView more correctly, passing it individual urwid.Text snippets, rather than a single large one. This hugely improves render time.
2011-03-15 00:05:33 +00:00
uLRUCache(),
Add HTTP body size limit specification to command-line tools.
2011-09-09 03:27:31 +00:00
u_parse_url(),
Refactor reverse proxying - Retain the specification from the Host header as a Request's description. - Expand upstream proxy specifications to include the scheme. We now say https://hostname:port - Move the "R" revert keybinding to "v" to make room for a reverse proxy binding that matches the command-line flag.
2012-02-18 01:45:22 +00:00
u_parse_proxy_spec(),
Firm up reverse proxy specification. - Extract proxy spec parsing and unparsing functions. - Add a status indicator in mitmproxy. - Add the "R" keybinding for changing the reverse proxy from within mitmproxy.
2012-02-18 03:27:09 +00:00
u_unparse_url(),
Expand test coverage.
2012-02-10 02:04:20 +00:00
u_parse_size(),
uclean_hanging_newline()
Initial checkin.
2010-02-16 04:09:07 +00:00
]
Reference in New Issue Copy Permalink