Grasscutters/mitmproxy
3
0
Fork 0
mirror of https://github.com/Grasscutters/mitmproxy.git synced 2024-11-27 02:24:18 +00:00
Code Issues Projects Releases Wiki Activity

Documentation.

Browse Source
This commit is contained in:
Aldo Cortesi 2012-06-24 10:54:37 +12:00
parent 3bc7f4e1cd
commit 6d0b49dfef
7 changed files with 361 additions and 327 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

300
README.mkd
View File

@ -1,4 +1,3 @@
__pathod__ is a collection of pathological tools for testing and torturing HTTP __pathod__ is a collection of pathological tools for testing and torturing HTTP
clients and servers. The project has three components: clients and servers. The project has three components:
@ -7,304 +6,6 @@ clients and servers. The project has three components:
- __libpathod.test__, an API for easily using __pathod__ and __pathoc__ in unit tests. - __libpathod.test__, an API for easily using __pathod__ and __pathoc__ in unit tests.
# pathod
At __pathod__'s heart is a tiny, terse language for crafting HTTP responses.
The simplest way to use __pathod__ is to fire up the daemon, and specify the
response behaviour you want using this language in the request URL. Here's a
minimal example:
http://localhost:9999/p/200
Everything after the "/p/" path component is a response specifier - in this
case just a vanilla 200 OK response. See the docs below to get (much) fancier.
You can also add anchors to the __pathod__ server that serve a fixed response
whenever a matching URL is requested:
pathod --anchor "/foo=200"
Here, "/foo" a regex specifying the anchor path, and the part after the "=" is
a response specifier.
__pathod__ also has a nifty built-in web interface, which lets you play with
the language by previewing responses, exposes activity logs, online help and
various other goodies. Try it by visiting the server root:
http://localhost:9999
## Specifying Responses
The general form of a response is as follows:
code[MESSAGE]:[colon-separated list of features]
Here's the simplest possible response specification, returning just an HTTP 200
OK message with no headers and no content:
200
We can embellish this a bit by specifying an optional custom HTTP response
message (if we don't, __pathod__ automatically creates an appropriate one). By
default for a 200 response code the message is "OK", but we can change it like
this:
200"YAY"
The quoted string here is an example of a Value Specifier, a syntax that is
used throughout the __pathod__ response specification language. In this case, the
quotes mean we're specifying a literal string, but there are many other fun
things we can do. For example, we can tell __pathod__ to generate 100k of random
ASCII letters instead:
200@100k,ascii_letters
Full documentation on the value specification syntax can be found in the next
section.
Following the response code specifier is a colon-separated list of features.
For instance, this specifies a response with a body consisting of 1 megabyte of
random data:
200:b@1m
And this is the same response with an ETag header added:
200:b@1m:h"Etag"="foo"
Both the header name and the header value are full value specifiers. Here's the
same response again, but with a 1k randomly generated header name:
200:b@1m:h@1k,ascii_letters="foo"
A few specific headers have shortcuts, because they're used so often. The
shortcut for the content-type header is "c":
200:b@1m:c"text/json"
That's it for the basic response definition. Now we can start mucking with the
responses to break clients. One common hard-to-test circumstance is hangs or
slow responses. __pathod__ has a pause operator that you can use to define
precisely when and how long the server should hang. Here, for instance, we hang
for 120 seconds after sending 50 bytes (counted from the first byte of the HTTP
response):
200:b@1m:p120,50
If that's not long enough, we can tell __pathod__ to hang forever:
200:b@1m:p120,f
Or to send all data, and then hang without disconnecting:
200:b@1m:p120,a
We can also ask __pathod__ to hang randomly:
200:b@1m:pr,a
There is a similar mechanism for dropping connections mid-response. So, we can
tell __pathod__ to disconnect after sending 50 bytes:
200:b@1m:d50
Or randomly:
200:b@1m:dr
All of these features can be combined. Here's a response that pauses twice,
once at 10 bytes and once at 20, then disconnects at 5000:
200:b@1m:p10,10:p20,10:d5000
## Features
#### hKEY=VALUE
Set a header. Both KEY and VALUE are full _Value Specifiers_.
#### bVALUE
Set the body. VALUE is a _Value Specifier_. When the body is set, __pathod__ will
automatically set the appropriate Content-Length header.
#### cVALUE
A shortcut for setting the Content-Type header. Equivalent to:
h"Content-Type"=VALUE
#### lVALUE
A shortcut for setting the Location header. Equivalent to:
h"Content-Type"=VALUE
#### dOFFSET
Disconnect after OFFSET bytes. The offset can also be "r", in which case __pathod__
will disconnect at a random point in the response.
#### pSECONDS,OFFSET
Pause for SECONDS seconds after OFFSET bytes. SECONDS can also be "f" to pause
forever. OFFSET can also be "r" to generate a random offset, or "a" for an
offset just after all data has been sent.
## Value Specifiers
There are three different flavours of value specification.
### Literal
Literal values are specified as a quoted strings:
"foo"
Either single or double quotes are accepted, and quotes can be escaped with
backslashes within the string:
'fo\'o'
### Files
You can load a value from a specified file path. To do so, you have to specify
a _staticdir_ option to __pathod__ on the command-line, like so:
pathod -d ~/myassets
All paths are relative paths under this directory. File loads are indicated by
starting the value specifier with the left angle bracket:
<my/path
The path value can also be a quoted string, with the same syntax as literals:
<"my/path"
### Generated values
An @-symbol lead-in specifies that generated data should be used. There are two
components to a generator specification - a size, and a data type. By default
__pathod__ assumes a data type of "bytes".
Here's a value specifier for generating 100 bytes:
@100
You can use standard suffixes to indicate larger values. Here, for instance, is
a specifier for generating 100 megabytes:
@100m
Data is generated and served efficiently - if you really want to send a
terabyte of data to a client, __pathod__ can do it. The supported suffixes are:
b = 1024**0 (bytes)
k = 1024**1 (kilobytes)
m = 1024**2 (megabytes)
g = 1024**3 (gigabytes)
t = 1024**4 (terabytes)
Data types are separated from the size specification by a comma. This
specification generates 100mb of ASCII:
@100m,ascii
Supported data types are:
ascii_letters
ascii_lowercase
ascii_uppercase
digits
hexdigits
letters
lowercase
octdigits
printable
punctuation
uppercase
whitespace
ascii
bytes
## API
__pathod__ exposes a simple API, intended to make it possible to drive and
inspect the daemon remotely for use in unit testing and the like.
### /api/log
Returns the current log buffer. At the moment the buffer size is 500 entries -
when the log grows larger than this, older entries are discarded. The returned
data is a JSON dictionary, with the form:
{
'logs': [ ENTRIES ]
}
Where each entry looks like this:
{
# Record of actions taken at specified byte offsets
'actions': [(200, 'disconnect'), (10, 'pause', 1)],
# HTTP return code
'code': 200,
# Request duration in seconds
'duration': 0.00020599365234375,
# ID unique to this invocation of pathod
'id': 2,
# The request that triggered the response
'request': {
'full_url': 'http://testing:9999/p/200:b@1000:p1,10:d200',
'headers': {
'Accept': '*/*',
'Host': 'localhost:9999',
'User-Agent': 'curl/7.21.4'
},
'host': 'localhost:9999',
'method': 'POST',
'path': '/p/200:b@1000:p1,10:d200',
'protocol': 'http',
'query': '',
'remote_address': ('10.0.0.234', 63448),
'uri': '/p/200:b@1000:p1,10:d200',
'version': 'HTTP/1.1'
},
# The response spec that was served. You can re-parse this to get full
# details on the response.
'spec': '200:b@1000:p1,10:d200',
# Time at which response startd.
'started': 1335735586.469218
}
You can preview the JSON data returned for a log entry through the built-in web
interface.
### /api/log/clear
A POST to this URL clears the log buffer.
# Installing # Installing
If you already have __pip__ on your system, installing __pathod__ and its If you already have __pip__ on your system, installing __pathod__ and its
@ -315,4 +16,3 @@ dependencies is dead simple:
The project uses the __nose__ unit testing framework, which you can get here: The project uses the __nose__ unit testing framework, which you can get here:
FIXME FIXME

30
README.txt
View File

@ -1,24 +1,18 @@
__pathod__ is a collection of pathological tools for testing and torturing HTTP
clients and servers. The project has three components:
**pathod** is a pathological HTTP/S daemon, useful for testing and torturing - __pathod__, an pathological HTTP daemon.
HTTP clients. At **pathod**'s heart is a tiny, terse language for crafting HTTP - __pathoc__, a perverse HTTP client.
responses. The simplest way to use **pathod** is to fire up the daemon, and - __libpathod.test__, an API for easily using __pathod__ and __pathoc__ in unit tests.
specify the response behaviour you want using this language in the request URL.
Here's a minimal example:
http://localhost:9999/p/200
Everything after the "/p/" path component is a response specifier - in this # Installing
case just a vanilla 200 OK response. See the complete docs to get (much)
fancier. You can also add anchors to the **pathod** server that serve a fixed
response whenever a matching URL is requested:
pathod --anchor "/foo=200" If you already have __pip__ on your system, installing __pathod__ and its
dependencies is dead simple:
pip install pathod
Here, "/foo" a regex specifying the anchor path, and the part after the "=" is The project uses the __nose__ unit testing framework, which you can get here:
a response specifier.
**pathod** also has a nifty built-in web interface, which lets you play with FIXME
the language by previewing responses, exposes activity logs, online help and
various other goodies. Try it by visiting the server root:
http://localhost:9999

6
doc-src/_layout_full.html
View File

@ -40,9 +40,9 @@
</a> </a>
<ul class="dropdown-menu"> <ul class="dropdown-menu">
<li><a href="@!top!@/index.html">pathod</a></li> <li><a href="@!top!@/pathod.html">pathod</a></li>
<li><a href="@!top!@/index.html">pathoc</a></li> <li><a href="@!top!@/pathoc.html">pathoc</a></li>
<li><a href="@!top!@/index.html">libpathod.test</a></li> <li><a href="@!top!@/test.html">libpathod.test</a></li>
</ul> </ul>
</li> </li>
</ul> </ul>

9
doc-src/index.py
View File

@ -8,11 +8,8 @@ this.titlePrefix = ""
this.site_url = "http://corte.si" this.site_url = "http://corte.si"
pages = [ pages = [
Page("index.html", "overview", namespace=dict(section="index")), Page("index.html", "overview", namespace=dict(section="index")),
Page("docs.html", "docs", namespace=dict(section="docs")), Page("pathod.html", "pathod", namespace=dict(section="docs")),
Page("pathoc.html", "pathoc", namespace=dict(section="docs")),
Page("test.html", "libpathod.test", namespace=dict(section="docs")),
sitemap.Sitemap("sitemap.xml") sitemap.Sitemap("sitemap.xml")
] ]
ns.sidebar = widgets.SiblingPageIndex(
pages[0],
depth=1,
divclass="sidebarmenu"
)

0
doc-src/docs.html → doc-src/pathoc.html
View File

338
doc-src/pathod.html Normal file
View File

@ -0,0 +1,338 @@
# pathod
At __pathod__'s heart is a tiny, terse language for crafting HTTP responses,
designed to be easy to specify in a request URL. The simplest way to use
__pathod__ is to fire up the daemon, and specify the response behaviour you
want using this language in the request URL. Here's a minimal example:
http://localhost:9999/p/200
Everything after the "/p/" path component is a response specifier - in this
case just a vanilla 200 OK response. See the docs below to get (much) fancier.
You can also add anchors to the __pathod__ server that serve a fixed response
whenever a matching URL is requested:
pathod --anchor "/foo=200"
Here, "/foo" a regex specifying the anchor path, and the part after the "=" is
a response specifier.
__pathod__ also has a nifty built-in web interface, which lets you play with
the language by previewing responses, exposes activity logs, online help and
various other goodies. Try it by visiting the server root:
http://localhost:9999
# Specifying Responses
The general form of a response is as follows:
code[MESSAGE]:[colon-separated list of features]
Here's the simplest possible response specification, returning just an HTTP 200
OK message with no headers and no content:
200
We can embellish this a bit by specifying an optional custom HTTP response
message (if we don't, __pathod__ automatically creates an appropriate one). By
default for a 200 response code the message is "OK", but we can change it like
this:
200"YAY"
The quoted string here is an example of a <a href=#valuespec>Value
Specifier</a>, a syntax that is used throughout the __pathod__ response
specification language. In this case, the quotes mean we're specifying a
literal string, but there are many other fun things we can do. For example, we
can tell __pathod__ to generate 100k of random ASCII letters instead:
200@100k,ascii_letters
Full documentation on the value specification syntax can be found in the next
section.
Following the response code specifier is a colon-separated list of features.
For instance, this specifies a response with a body consisting of 1 megabyte of
random data:
200:b@1m
And this is the same response with an ETag header added:
200:b@1m:h"Etag"="foo"
Both the header name and the header value are full value specifiers. Here's the
same response again, but with a 1k randomly generated header name:
200:b@1m:h@1k,ascii_letters="foo"
A few specific headers have shortcuts, because they're used so often. The
shortcut for the content-type header is "c":
200:b@1m:c"text/json"
That's it for the basic response definition. Now we can start mucking with the
responses to break clients. One common hard-to-test circumstance is hangs or
slow responses. __pathod__ has a pause operator that you can use to define
precisely when and how long the server should hang. Here, for instance, we hang
for 120 seconds after sending 50 bytes (counted from the first byte of the HTTP
response):
200:b@1m:p120,50
If that's not long enough, we can tell __pathod__ to hang forever:
200:b@1m:p120,f
Or to send all data, and then hang without disconnecting:
200:b@1m:p120,a
We can also ask __pathod__ to hang randomly:
200:b@1m:pr,a
There is a similar mechanism for dropping connections mid-response. So, we can
tell __pathod__ to disconnect after sending 50 bytes:
200:b@1m:d50
Or randomly:
200:b@1m:dr
All of these features can be combined. Here's a response that pauses twice,
once at 10 bytes and once at 20, then disconnects at 5000:
200:b@1m:p10,10:p20,10:d5000
## Response Features
<table class="table table-bordered table-condensed">
<tbody >
<tr>
<td>
hKEY=VALUE
</td>
<td>
Set a header. Both KEY and VALUE are full <a href=#valuespec>Value Specifiers</a>.
</td>
</tr>
<tr>
<td>
bVALUE
</td>
<td>
Set the body. VALUE is a <a href=#valuespec>Value
Specifier</a>. When the body is set, __pathod__ will
automatically set the appropriate Content-Length header.
</td>
</tr>
<tr>
<td>
cVALUE
</td>
<td>
A shortcut for setting the Content-Type header. Equivalent to:
<pre>h"Content-Type"=VALUE</pre>
</td>
</tr>
<tr>
<td>
lVALUE
</td>
<td>
A shortcut for setting the Location header. Equivalent to:
<pre>h"Content-Type"=VALUE</pre>
</td>
</tr>
<tr>
<td>
dOFFSET
</td>
<td>
Disconnect after OFFSET bytes. The offset can also be "r", in which case __pathod__
will disconnect at a random point in the response.
</td>
</tr>
<tr>
<td>
pSECONDS,OFFSET
</td>
<td>
Pause for SECONDS seconds after OFFSET bytes. SECONDS can also be "f" to pause
forever. OFFSET can also be "r" to generate a random offset, or "a" for an
offset just after all data has been sent.
</td>
</tr>
</tbody>
</table>
<a id="valuespec"></a>
## VALUE Specifiers
There are three different flavours of value specification.
### Literal
Literal values are specified as a quoted strings:
"foo"
Either single or double quotes are accepted, and quotes can be escaped with
backslashes within the string:
'fo\'o'
### Files
You can load a value from a specified file path. To do so, you have to specify
a _staticdir_ option to __pathod__ on the command-line, like so:
pathod -d ~/myassets
All paths are relative paths under this directory. File loads are indicated by
starting the value specifier with the left angle bracket:
<my/path
The path value can also be a quoted string, with the same syntax as literals:
<"my/path"
### Generated values
An @-symbol lead-in specifies that generated data should be used. There are two
components to a generator specification - a size, and a data type. By default
__pathod__ assumes a data type of "bytes".
Here's a value specifier for generating 100 bytes:
@100
You can use standard suffixes to indicate larger values. Here, for instance, is
a specifier for generating 100 megabytes:
@100m
Data is generated and served efficiently - if you really want to send a
terabyte of data to a client, __pathod__ can do it. The supported suffixes are:
<table class="table table-bordered table-condensed">
<tbody >
<tr>
<td>b</td> <td>1024**0 (bytes)</td>
</tr>
<tr>
<td>k</td> <td>1024**1 (kilobytes)</td>
</tr>
<tr>
<td>m</td> <td>1024**2 (megabytes)</td>
</tr>
<tr>
<td>g</td> <td>1024**3 (gigabytes)</td>
</tr>
<tr>
<td>t</td> <td>1024**4 (terabytes)</td>
</tr>
</tbody>
</table>
Data types are separated from the size specification by a comma. This
specification generates 100mb of ASCII:
@100m,ascii
Supported data types are:
- ascii_letters
- ascii_lowercase
- ascii_uppercase
- digits
- hexdigits
- letters
- lowercase
- octdigits
- printable
- punctuation
- uppercase
- whitespace
- ascii
- bytes
# API
__pathod__ exposes a simple API, intended to make it possible to drive and
inspect the daemon remotely for use in unit testing and the like.
### /api/log
Returns the current log buffer. At the moment the buffer size is 500 entries -
when the log grows larger than this, older entries are discarded. The returned
data is a JSON dictionary, with the form:
{
'logs': [ ENTRIES ]
}
Where each entry looks like this:
{
# Record of actions taken at specified byte offsets
'actions': [(200, 'disconnect'), (10, 'pause', 1)],
# HTTP return code
'code': 200,
# Request duration in seconds
'duration': 0.00020599365234375,
# ID unique to this invocation of pathod
'id': 2,
# The request that triggered the response
'request': {
'full_url': 'http://testing:9999/p/200:b@1000:p1,10:d200',
'headers': {
'Accept': '*/*',
'Host': 'localhost:9999',
'User-Agent': 'curl/7.21.4'
},
'host': 'localhost:9999',
'method': 'POST',
'path': '/p/200:b@1000:p1,10:d200',
'protocol': 'http',
'query': '',
'remote_address': ('10.0.0.234', 63448),
'uri': '/p/200:b@1000:p1,10:d200',
'version': 'HTTP/1.1'
},
# The response spec that was served. You can re-parse this to get full
# details on the response.
'spec': '200:b@1000:p1,10:d200',
# Time at which response startd.
'started': 1335735586.469218
}
You can preview the JSON data returned for a log entry through the built-in web
interface.
### /api/log/clear
A POST to this URL clears the log buffer.

5
doc-src/test.html Normal file
View File

@ -0,0 +1,5 @@
Test.