@@ -39,7 +39,7 @@ information mitmproxy needs to proceed.
CONNECT 10.1.1.1:443 HTTP/1.1-Using the IP address is perfectly legitimate because it gives us enough -information to initiate the pipe, even though it doesn't reveal the remote -hostname. +Using the IP address is perfectly legitimate because it gives us enough information to initiate the pipe, even though it doesn't reveal the remote hostname. -Mitmproxy has a cunning mechanism that smooths this over - [upstream -certificate sniffing](@!urlTo("features/upstreamcerts.html")!@). As soon as we -see the CONNECT request, we pause the client part of the conversation, and -initiate a simultaneous connection to the server. We complete the SSL handshake -with the server, and inspect the certificates it used. Now, we use the Common -Name in the upstream SSL certificates to generate the dummy certificate for the -client. Voila, we have the correct hostname to present to the client, even if -it was never specified. +Mitmproxy has a cunning mechanism that smooths this over - [upstream certificate sniffing](@!urlTo("features/upstreamcerts.html")!@). As soon as we see the CONNECT request, we pause the client part of the conversation, and initiate a simultaneous connection to the server. We complete the SSL handshake with the server, and inspect the certificates it used. Now, we use the Common Name in the upstream SSL certificates to generate the dummy certificate for the client. Voila, we have the correct hostname to present to the client, even if it was never specified. ## Complication 2: Subject Alternative Name @@ -127,56 +95,33 @@ them to the generated dummy certificate. ## Complication 3: Server Name Indication -One of the big limitations of conventional SSL is that each certificate -requires its own IP address. This means that you couldn't do virtual hosting -where multiple domains with independent certificates share the same IP address. -In a world with a rapidly shrinking IPv4 address pool this is a problem, and we -have a solution in the form of the [Server Name -Indication](http://en.wikipedia.org/wiki/Server_Name_Indication) extension to -the SSL and TLS protocols. This lets the client specify the remote server name -at the start of the SSL handshake, which then lets the server select the right -certificate to complete the process. +One of the big limitations of vanilla SSL is that each certificate requires its own IP address. This means that you couldn't do virtual hosting where multiple domains with independent certificates share the same IP address. In a world with a rapidly shrinking IPv4 address pool this is a problem, and we have a solution in the form of the [Server Name Indication](http://en.wikipedia.org/wiki/Server_Name_Indication) extension to the SSL and TLS protocols. This lets the client specify the remote server name at the start of the SSL handshake, which then lets the server select the right certificate to complete the process. -SNI breaks our upstream certificate sniffing process, because when we connect -without using SNI, we get served a default certificate that may have nothing to -do with the certificate expected by the client. The solution is another tricky -complication to the client connection process. After the client connects, we -allow the SSL handshake to continue until just _after_ the SNI value has been -passed to us. Now we can pause the conversation, and initiate an upstream -connection using the correct SNI value, which then serves us the correct -upstream certificate, from which we can extract the expected CN and SANs. - -There's another wrinkle here. Due to a limitation of the SSL library mitmproxy -uses, we can't detect that a connection _hasn't_ sent an SNI request until it's -too late for upstream certificate sniffing. In practice, we therefore make a -vanilla SSL connection upstream to sniff non-SNI certificates, and then discard -the connection if the client sends an SNI notification. If you're watching your -traffic with a packet sniffer, you'll see two connections to the server when an -SNI request is made, the first of which is immediately closed after the SSL -handshake. Luckily, this is almost never an issue in practice. +SNI breaks our upstream certificate sniffing process, because when we connect without using SNI, we get served a default certificate that may have nothing to do with the certificate expected by the client. The solution is another tricky complication to the client connection process. After the client connects, we allow the SSL handshake to continue until just _after_ the SNI value has been passed to us. Now we can pause the conversation, and initiate an upstream connection using the correct SNI value, which then serves us the correct upstream certificate, from which we can extract the expected CN and SANs. +There's another wrinkle here. Due to a limitation of the SSL library mitmproxy uses, we can't detect that a connection _hasn't_ sent an SNI request until it's too late for upstream certificate sniffing. In practice, we therefore make a vanilla SSL connection upstream to sniff non-SNI certificates, and then discard the connection if the client sends an SNI notification. If you're watching your traffic with a packet sniffer, you'll see two connections to the server when an SNI request is made, the first of which is immediately closed after the SSL handshake. Luckily, this is almost never an issue in practice. ## Putting it all together -Lets put all of this together into the complete explicitly proxied HTTPS flow. +Lets put all of this together into the complete explicitly proxied HTTPS flow.
| 1 | +1 | The client makes a connection to mitmproxy, and issues an HTTP CONNECT request. | ||||||||||||||||||||||||||||||||||||
| 2 | +2 | Mitmproxy responds with a 200 Connection Established, as if it has set up the CONNECT pipe. | ||||||||||||||||||||||||||||||||||||
| 3 | +3 | The client believes it's talking to the remote server, and initiates the SSL connection. It uses SNI to indicate the hostname @@ -184,33 +129,33 @@ Lets put all of this together into the complete explicitly proxied HTTPS flow. | ||||||||||||||||||||||||||||||||||||
| 4 | +4 | Mitmproxy connects to the server, and establishes an SSL connection using the SNI hostname indicated by the client. | - +||||||||||||||||||||||||||||||||||||
| 5 | +5 | The server responds with the matching SSL certificate, which contains the CN and SAN values needed to generate the interception certificate. | ||||||||||||||||||||||||||||||||||||
| 6 | +6 | Mitmproxy generates the interception cert, and continues the client SSL handshake paused in step 3. | ||||||||||||||||||||||||||||||||||||
| 7 | +7 | The client sends the request over the established SSL connection. | ||||||||||||||||||||||||||||||||||||
| 7 | +7 | Mitmproxy passes the request on to the server over the SSL connection initiated in step 4. | @@ -234,11 +179,11 @@ redirection mechanism that transparently reroutes a TCP connection destined for a server on the Internet to a listening proxy server. This usually takes the form of a firewall on the same host as the proxy server - [iptables](http://www.netfilter.org/) on Linux or -[pf](http://en.wikipedia.org/wiki/PF_(firewall)) on OSX. Once the client has +[pf](http://en.wikipedia.org/wiki/PF_\(firewall\)) on OSX. Once the client has initiated the connection, it makes a vanilla HTTP request, which might look something like this: -
| 1 | +1 | The client makes a connection to the server. |
| 2 | +2 | The router redirects the connection to mitmproxy, which is typically listening on a local port of the same host. Mitmproxy @@ -270,16 +215,16 @@ this information, the process is fairly straight-forward. destination was. |
| 3 | +3 | Now, we simply read the client's request... |
| 4 | +4 | ... and forward it upstream. | - +
| 1 | +1 | The client makes a connection to the server. |
| 2 | +2 | The router redirects the connection to mitmproxy, which is typically listening on a local port of the same host. Mitmproxy @@ -312,7 +257,7 @@ and cope with SNI. destination was. |
| 3 | +3 | The client believes it's talking to the remote server, and initiates the SSL connection. It uses SNI to indicate the hostname @@ -320,33 +265,33 @@ and cope with SNI. |
| 4 | +4 | Mitmproxy connects to the server, and establishes an SSL connection using the SNI hostname indicated by the client. | - +
| 5 | +5 | The server responds with the matching SSL certificate, which contains the CN and SAN values needed to generate the interception certificate. |
| 6 | +6 | Mitmproxy generates the interception cert, and continues the client SSL handshake paused in step 3. |
| 7 | +7 | The client sends the request over the established SSL connection. |
| 7 | +7 | Mitmproxy passes the request on to the server over the SSL connection initiated in step 4. | diff --git a/doc-src/transparent.html b/doc-src/transparent.html index b28b04f64..689a28425 100644 --- a/doc-src/transparent.html +++ b/doc-src/transparent.html @@ -1,3 +1,15 @@ +When a transparent proxy is used, traffic is redirected into a proxy at the network layer, without +any client configuration being required. This makes transparent proxying ideal for those situations +where you can't change client behaviour - proxy-oblivious Android applications being a common +example. +To set up transparent proxying, we need two new components. The first is a +redirection mechanism that transparently reroutes a TCP connection destined for +a server on the Internet to a listening proxy server. This usually takes the +form of a firewall on the same host as the proxy server - +[iptables](http://www.netfilter.org/) on Linux or +[pf](http://en.wikipedia.org/wiki/PF_\(firewall\)) on OSX. When the proxy receives a redirected connection, it sees a vanilla HTTP request, without a host specification. This is where the second new component comes in - a host module that allows us to query the redirector for the original destination of the TCP connection. + +At the moment, mitmproxy supports transparent proxying on OSX Lion and above, and all current flavors of Linux.kkkkk \ No newline at end of file diff --git a/doc-src/transparent/osx.html b/doc-src/transparent/osx.html index 201588737..77eea63b4 100644 --- a/doc-src/transparent/osx.html +++ b/doc-src/transparent/osx.html @@ -20,7 +20,7 @@ OSX.