Grasscutters/mitmproxy
3
0
Fork 0
mirror of https://github.com/Grasscutters/mitmproxy.git synced 2024-11-27 02:24:18 +00:00
Code Issues Projects Releases Wiki Activity
17facd8b72
mitmproxy/test/test_utils.py

250 lines
6.7 KiB
Python
Raw Normal View History

JSON pretty-printing. Also rename the display modes ("pretty" instead of "indent"), and expand the built-in documentation.
2011-06-30 01:27:27 +00:00
import textwrap, cStringIO, os, time, re, json
Initial checkin.
2010-02-16 04:09:07 +00:00
import libpry
from libmproxy import utils
Add a hideous kludge to fix not-yet-valid certificates. - The OpenSSL x509 has no way to explicitly set the notBefore value on certificates. - If two systems have the same configured time, it's possible to return a certificate before the validity start time has arrived. - We "solve" this by waiting for one second when a certificate is first generated before returning the cert. The alternative is to rewrite pretty much all of our certificate generation, a thought too horrible to contemplate.
2011-06-11 03:16:16 +00:00
utils.CERT_SLEEP_TIME = 0
Initial checkin.
2010-02-16 04:09:07 +00:00
Add timestamps to flows. For now, these are only displayed on the connection view screen, with second granularity.
2011-02-03 00:30:47 +00:00
class uformat_timestamp(libpry.AutoTree):
def test_simple(self):
Store timestamps on flow components as a UTC time tuple. Format is: (tm_year,tm_mon,tm_mday,tm_hour,tm_min, tm_sec,tm_wday,tm_yday,tm_isdst)
2011-03-07 00:46:02 +00:00
assert utils.format_timestamp(utils.timestamp())
Add timestamps to flows. For now, these are only displayed on the connection view screen, with second granularity.
2011-02-03 00:30:47 +00:00
Initial checkin.
2010-02-16 04:09:07 +00:00
class uisBin(libpry.AutoTree):
def test_simple(self):
assert not utils.isBin("testing\n\r")
assert utils.isBin("testing\x01")
assert utils.isBin("testing\x0e")
assert utils.isBin("testing\x7f")
Try not to hang when user views large request & response bodies Two different strategies here: - Use a simple heuristic to detect if we're looking at XML data when indent mode is used. On non-XML data we can hang even on small documents. - Only view partial data for large bodies. At the moment the cutoff is 100k. I might finetune this later.
2011-06-27 03:59:17 +00:00
class uisXML(libpry.AutoTree):
def test_simple(self):
assert not utils.isXML("foo")
assert utils.isXML("<foo")
assert utils.isXML(" \n<foo")
Initial checkin.
2010-02-16 04:09:07 +00:00
class uhexdump(libpry.AutoTree):
def test_simple(self):
assert utils.hexdump("one\0"*10)
Unit test++
2011-08-02 04:52:47 +00:00
class udel_all(libpry.AutoTree):
def test_simple(self):
d = dict(a=1, b=2, c=3)
utils.del_all(d, ["a", "x", "b"])
assert d.keys() == ["c"]
Initial checkin.
2010-02-16 04:09:07 +00:00
class upretty_size(libpry.AutoTree):
def test_simple(self):
assert utils.pretty_size(100) == "100B"
assert utils.pretty_size(1024) == "1kB"
Code cleanliness - appease pychecker.
2011-08-03 22:34:34 +00:00
assert utils.pretty_size(1024 + (1024/2.0)) == "1.5kB"
Initial checkin.
2010-02-16 04:09:07 +00:00
assert utils.pretty_size(1024*1024) == "1M"
class uData(libpry.AutoTree):
def test_nonexistent(self):
General cleanup. Cut out unused variables and code, generally shut up pychecker as much as is reasonable.
2011-08-02 04:14:33 +00:00
libpry.raises("does not exist", utils.pkg_data.path, "nonexistent")
Initial checkin.
2010-02-16 04:09:07 +00:00
Rip out BeautifulSoup, and use a custom XML-ish prettyprinter.
2011-02-06 01:17:30 +00:00
class upretty_xmlish(libpry.AutoTree):
def test_tagre(self):
def f(s):
return re.search(utils.TAG, s, re.VERBOSE|re.MULTILINE)
assert f(r"<body>")
assert f(r"<body/>")
assert f(r"< body/>")
assert f(r"< body/ >")
assert f(r"< body / >")
assert f(r"<foo a=b>")
assert f(r"<foo a='b'>")
assert f(r"<foo a='b\"'>")
assert f(r'<a b=(a.b) href="foo">')
assert f('<td width=25%>')
Further fine-tuning for pretty_xmlish.
2011-02-06 03:56:13 +00:00
assert f('<form name="search" action="/search.php" method="get" accept-charset="utf-8" class="search">')
assert f('<img src="gif" width="125" height="16" alt=&quot;&quot; />')
Rip out BeautifulSoup, and use a custom XML-ish prettyprinter.
2011-02-06 01:17:30 +00:00
Basix XML/HTML pretty-printing in flow viewer.
2011-01-27 04:26:01 +00:00
def test_all(self):
Rip out BeautifulSoup, and use a custom XML-ish prettyprinter.
2011-02-06 01:17:30 +00:00
def isbalanced(ret):
# The last tag should have no indent
assert ret[-1].strip() == ret[-1]
s = "<html><br><br></br><p>one</p></html>"
ret = utils.pretty_xmlish(s)
isbalanced(ret)
s = r"""
<body bgcolor=#ffffff text=#000000 link=#0000cc vlink=#551a8b alink=#ff0000 onload="document.f.q.focus();if(document.images)new Image().src='/images/srpr/nav_logo27.png'" ><textarea id=csi style=display:none></textarea></body>
"""
isbalanced(utils.pretty_xmlish(textwrap.dedent(s)))
s = r"""
<a href="http://foo.com" target="">
<img src="http://foo.gif" alt="bar" height="25" width="132">
</a>
"""
isbalanced(utils.pretty_xmlish(textwrap.dedent(s)))
s = r"""
<!DOCTYPE html PUBLIC \"-//W3C//DTD XHTML 1.0 Strict//EN\"
\"http://www.w3.org/TR/xhtml1/DTD/xhtml1-strict.dtd\">
<html></html>
"""
ret = utils.pretty_xmlish(textwrap.dedent(s))
isbalanced(ret)
s = "<html><br/><p>one</p></html>"
ret = utils.pretty_xmlish(s)
assert len(ret) == 6
isbalanced(ret)
s = "gobbledygook"
Further fine-tuning for pretty_xmlish.
2011-02-06 03:56:13 +00:00
assert utils.pretty_xmlish(s) == ["gobbledygook"]
Rip out BeautifulSoup, and use a custom XML-ish prettyprinter.
2011-02-06 01:17:30 +00:00
Basix XML/HTML pretty-printing in flow viewer.
2011-01-27 04:26:01 +00:00
JSON pretty-printing. Also rename the display modes ("pretty" instead of "indent"), and expand the built-in documentation.
2011-06-30 01:27:27 +00:00
class upretty_json(libpry.AutoTree):
def test_one(self):
s = json.dumps({"foo": 1})
assert utils.pretty_json(s)
assert not utils.pretty_json("moo")
Add a pretty-printing mode for urlencoded form data.
2011-07-15 04:16:43 +00:00
class u_urldecode(libpry.AutoTree):
def test_one(self):
s = "one=two&three=four"
assert len(utils.urldecode(s)) == 2
Clean up certificate generation. - Use templates for config files. We can re-introduce customization of the certificate attributes when we need them. - Split CA and cert generation into separate functions. - Generation methods provide an error return when generation fails. - When the user explicitly specifies a certificate, we don't generate it, but fail if it doesn't exist.
2011-02-19 23:12:55 +00:00
class udummy_ca(libpry.AutoTree):
def test_all(self):
d = self.tmpdir()
path = os.path.join(d, "foo/cert.cnf")
assert utils.dummy_ca(path)
assert os.path.exists(path)
Revamp key generation. We now create three different files in the .mitmproxy directory when a dummy CA is made: mitmproxy-ca.pem - the CA, including private key mitmproxy-ca-cert.p12 - A pkcs12 version of the certificate, for distribution to Windows. mitmproxy-ca-cert.pem - A PEM version of the certificate, for distribution to everyone else.
2011-03-18 03:45:31 +00:00
path = os.path.join(d, "foo/cert2.pem")
assert utils.dummy_ca(path)
assert os.path.exists(path)
assert os.path.exists(os.path.join(d, "foo/cert2-cert.pem"))
assert os.path.exists(os.path.join(d, "foo/cert2-cert.p12"))
Clean up certificate generation. - Use templates for config files. We can re-introduce customization of the certificate attributes when we need them. - Split CA and cert generation into separate functions. - Generation methods provide an error return when generation fails. - When the user explicitly specifies a certificate, we don't generate it, but fail if it doesn't exist.
2011-02-19 23:12:55 +00:00
class udummy_cert(libpry.AutoTree):
def test_with_ca(self):
d = self.tmpdir()
cacert = os.path.join(d, "foo/cert.cnf")
assert utils.dummy_ca(cacert)
assert utils.dummy_cert(
os.path.join(d, "foo"),
cacert,
"foo.com"
)
assert os.path.exists(os.path.join(d, "foo", "foo.com.pem"))
Revamp SSL configuration. - Move option parsing utiliities to proxy.py - Don't have a global config object. Pass it as an argument to ProxyServer. - Simplify certificate generation logic.
2011-02-19 23:53:42 +00:00
# Short-circuit
assert utils.dummy_cert(
os.path.join(d, "foo"),
cacert,
"foo.com"
)
Clean up certificate generation. - Use templates for config files. We can re-introduce customization of the certificate attributes when we need them. - Split CA and cert generation into separate functions. - Generation methods provide an error return when generation fails. - When the user explicitly specifies a certificate, we don't generate it, but fail if it doesn't exist.
2011-02-19 23:12:55 +00:00
def test_no_ca(self):
d = self.tmpdir()
assert utils.dummy_cert(
d,
None,
"foo.com"
)
assert os.path.exists(os.path.join(d, "foo.com.pem"))
Improve responsiveness of request and response viewing. - Computing the view of a large body is expensive, so we introduce an LRU cache to hold the latest 20 results. - Use ListView more correctly, passing it individual urwid.Text snippets, rather than a single large one. This hugely improves render time.
2011-03-15 00:05:33 +00:00
class uLRUCache(libpry.AutoTree):
def test_one(self):
class Foo:
ran = False
@utils.LRUCache(2)
def one(self, x):
self.ran = True
return x
f = Foo()
assert f.one(1) == 1
assert f.ran
f.ran = False
assert f.one(1) == 1
assert not f.ran
f.ran = False
assert f.one(1) == 1
assert not f.ran
assert f.one(2) == 2
assert f.one(3) == 3
assert f.ran
f.ran = False
assert f.one(1) == 1
assert f.ran
assert len(f._cached_one) == 2
assert len(f._cachelist_one) == 2
Make pretty-printing more robust. Also, since BeautifulSoup is so damn slow, print a statusbar message saying that we're calculating a pretty version of the response. Maybe I should add hangman or something, becuase on a 200k document this can take ages.
2011-01-27 23:07:27 +00:00
Move all HTTP objects to flow.py That's Request, Response, ClientConnect, ClientDisconnect, Error, and Headers.
2011-08-03 10:38:23 +00:00
class u_parse_url(libpry.AutoTree):
def test_simple(self):
assert not utils.parse_url("")
u = "http://foo.com:8888/test"
s, h, po, pa = utils.parse_url(u)
assert s == "http"
assert h == "foo.com"
assert po == 8888
assert pa == "/test"
s, h, po, pa = utils.parse_url("http://foo/bar")
assert s == "http"
assert h == "foo"
assert po == 80
assert pa == "/bar"
s, h, po, pa = utils.parse_url("http://foo")
assert pa == "/"
s, h, po, pa = utils.parse_url("https://foo")
assert po == 443
Improve robustness against invalid data.
2011-09-04 19:47:47 +00:00
assert not utils.parse_url("https://foo:bar")
assert not utils.parse_url("https://foo:")
Add HTTP body size limit specification to command-line tools.
2011-09-09 03:27:31 +00:00
class u_parse_size(libpry.AutoTree):
def test_simple(self):
assert utils.parse_size("1") == 1
assert utils.parse_size("1k") == 1024
assert utils.parse_size("1m") == 1024**2
assert utils.parse_size("1g") == 1024**3
libpry.raises(ValueError, utils.parse_size, "1f")
libpry.raises(ValueError, utils.parse_size, "ak")
Initial checkin.
2010-02-16 04:09:07 +00:00
tests = [
Add timestamps to flows. For now, these are only displayed on the connection view screen, with second granularity.
2011-02-03 00:30:47 +00:00
uformat_timestamp(),
Initial checkin.
2010-02-16 04:09:07 +00:00
uisBin(),
Try not to hang when user views large request & response bodies Two different strategies here: - Use a simple heuristic to detect if we're looking at XML data when indent mode is used. On non-XML data we can hang even on small documents. - Only view partial data for large bodies. At the moment the cutoff is 100k. I might finetune this later.
2011-06-27 03:59:17 +00:00
uisXML(),
Initial checkin.
2010-02-16 04:09:07 +00:00
uhexdump(),
upretty_size(),
uData(),
Rip out BeautifulSoup, and use a custom XML-ish prettyprinter.
2011-02-06 01:17:30 +00:00
upretty_xmlish(),
JSON pretty-printing. Also rename the display modes ("pretty" instead of "indent"), and expand the built-in documentation.
2011-06-30 01:27:27 +00:00
upretty_json(),
Add a pretty-printing mode for urlencoded form data.
2011-07-15 04:16:43 +00:00
u_urldecode(),
Unit test++
2011-08-02 04:52:47 +00:00
udel_all(),
Clean up certificate generation. - Use templates for config files. We can re-introduce customization of the certificate attributes when we need them. - Split CA and cert generation into separate functions. - Generation methods provide an error return when generation fails. - When the user explicitly specifies a certificate, we don't generate it, but fail if it doesn't exist.
2011-02-19 23:12:55 +00:00
udummy_ca(),
udummy_cert(),
Improve responsiveness of request and response viewing. - Computing the view of a large body is expensive, so we introduce an LRU cache to hold the latest 20 results. - Use ListView more correctly, passing it individual urwid.Text snippets, rather than a single large one. This hugely improves render time.
2011-03-15 00:05:33 +00:00
uLRUCache(),
Add HTTP body size limit specification to command-line tools.
2011-09-09 03:27:31 +00:00
u_parse_url(),
u_parse_size()
Initial checkin.
2010-02-16 04:09:07 +00:00
]
Reference in New Issue Copy Permalink