90% less spam in Hotmail, 15% less spam on the Internet

In our war on spam, we’re making real progress. We’ve cut spam in Hotmail inboxes by 90% from its peak. We’ve played a key role in reducing spam on the Internet by 15% from its peak. And we’ve made it harder for spammers to use Hotmail to send spam – reducing “outbound spam” from Hotmail by 75%.

Last year, we wrote about how Hotmail was fighting a war on spam with our SmartScreen™ technology. This post gives an update on the latest and greatest features and innovations that we’ve brought to bear against the spammers. We’ve made it so hard on the spammers that they have now turned to a technique called “reputation hijacking.” I’ll explain how spammers use reputation hijacking across all email services and how Hotmail is shutting them down.

As you’ll recall from our earlier posts, spam is a huge problem that continues to plague the Internet. Historically, more than 90% of all email sent has been spam, and spam affects every email provider. Spammers do what they do because it’s profitable; they need only a few people to click on the spam messages in order to make money.

Way back in 2006, Hotmail had a big spam problem, and we got a deservedly bad reputation for it. Since then, we’ve made amazing advances, and over the last few years, we’ve wrestled the spammers to the ground. Here’s a chart that shows the amount of Spam In The Inbox (SITI) for Hotmail users over the last several years, compared with the amount of spam on the Internet (expressed as a percentage of all email that is sent on the Internet).



The chart shows two things:

Hotmail keeps spam out of your Inbox
We’ve reduced the level of spam in Hotmail by 90% since its peak in 2006. Since last year, we’ve reduced what was left by another 40% (from 5% true SITI to 3% true SITI).

We’ve helped to reduce overall spam on the Internet
The percentage of spam on the Internet has actually declined 15% from its peak in 2008, due to a number of factors including the legal and technical disruptive action Microsoft has helped drive in the prosecution of spammers and the takedowns of botnets used to send spam. Botnets – collections of people’s malware-infected computers covertly operating under the remote control of a cybercriminal – are often used to send spam (and commit other online crimes). This video explains a little more about how botnets are used to send spam.



Microsoft is working with law enforcement and others in the industry to proactively take down and dismantle botnets, including our recent takedowns of the Waledac and Rustock botnets. These disruptive actions are proving to be important in the fight against spam by taking away the tools and infrastructure cybercriminals use to spam the world. These efforts are paying off: before we took them down, Rustock was known as one of the largest single sources of spam on the Internet, capable of sending up to 30 billion spam messages a day. Global spam levels have gone down and stayed down since we took them out.

Our relentless pursuit and prosecution of spammers helps not only Hotmail, but all email users on the Internet. In fact, Microsoft has established a Digital Crimes Unit whose sole mission is to disrupt cybercrime like this. Spammers may keep developing new tactics and tools, but Hotmail and the Microsoft Digital Crimes Unit are going to keep working together on disruptive actions to help protect our customers and make the Internet safer for everyone.

Getting SITI low and keeping it low

Between 2006 and 2009, we dropped true SITI from 35% to under 5% with a variety of investments including connection-time filtering, content filtering, blocklist and safelist preferences, and more. Of course, the spammers continue to come and continue to get more and more clever. But we’ve not only held the spammers at bay, we’ve actually reduced SITI even more. Over the last year, we’ve dropped SITI to historically low rates – below 3%. Here are a couple of the new tools we’ve created to help us keep winning this fight:

Personalization
Our spam filters are great at filtering out spam for the general population. However, we knew we could do better. So we created personalized spam filters that work based on how you use email – using information about the people you send email to and receive it from and also which email messages you actually read.

Trusted sender
Hotmail helps you to visually identify trusted senders in your inbox, particularly banks and other institutions commonly used for phishing scams. We put safety logos next to only those senders that we recognize as legitimate so that you can more easily spot malicious imitators. It’s important to note that this also helps us take more aggressive spam-prevention action on email that is attempting to imitate a legitimate trusted sender.

These two tools augment the efficiency of our SmartScreen™ filters. But of course, we’re also continuously tuning the other SmartScreen™ features – like Time Travelling filter, IP reputation, URL reputation and more – to get additional gains in spam prevention.

Spammers use Hotmail, too

Almost nothing is more frustrating for us than knowing that the spammers use Hotmail, too. Of course, spammers use all the major email services to send spam, and all mail providers must battle the problem we call “outbound spam.” Outbound spam is a form of “reputation hijacking.” After all, Hotmail maintains a good reputation among all email providers; simply put, email from Hotmail gets delivered, and the spammers know that.

Just as we’ve made great strides battling inbound spam (SITI), we have also made it increasingly difficult for the spammers to use Hotmail as a spam-sending tool. In fact, over the last year, we’ve reduced the volume of outbound spam from Hotmail by 75%.

Here are a few of the innovative features that helped us get it done:

Account reputation
As you use your account, you gain a “reputation.” Good behavior (receiving email from the same people you sent email to, for example) gains you a good reputation. Bad behavior (sending a bunch of email and getting only delivery errors, for example) gains you a bad reputation, as these behaviors are indicative of spammers and other service abusers. Gain a bad enough rep, and we change the way your account works. For example, we will prevent accounts with bad reputations from sending mail.

Account creation limits
We have a variety of ways that we throttle account creation in order to prevent spammers from getting an unlimited number of free accounts to use in sending spam. For example, we limit the number of accounts that can be created per day from a particular IP address.

Outbound content filters
Just like we filter incoming mail to remove spam, we now filter outbound mail as well. For example, we look for suspicious content that matches known spam campaigns.

Spam and compromised accounts

In the old days (you know, two years ago), the spammers just opened email accounts at one of the major providers to send spam. After all, accounts at Yahoo!, Gmail, AOL, and Hotmail are free and can send email, which is pretty much all you need to start a spam campaign.

But with the advances we’ve made in account reputation, these accounts have become less and less useful to spammers. Unfortunately, our success in preventing new accounts from sending spam had a tragic side-effect: Spammers turned to using existing customer accounts to send spam. This is a second form of reputation hijacking, in which the spammers are hijacking your reputation as a good customer of Hotmail. How? By hijacking your account.

In fact, most outbound spam now comes from hijacked accounts.

As the problem of account hijacking has grown over the past few years, we’ve invested more and more energy into protecting your accounts and, in doing so, making this avenue of sending spam less and less attractive to the bad guys. We fight account hijacking by focusing on three key activities:

Detection
When a spammer hijacks an account, we have many ways of detecting that hijacking. We look for unusual behavior from the account, including access from unusual IP addresses, sending an unusual volume of mail, sending mail that triggers our outbound spam filters, etc. We even introduced a feature that lets you report your friends if their accounts get hijacked.

Remediation
Once we’ve identified an account as compromised, we want to block the hijacker from accessing the account and then return the account to the rightful owner as painlessly as possible. We typically block the account and then send the real account owner through an account recovery flow that the bad guys will have difficulty getting through. We provide many ways for you to protect your account by setting up “proofs” that only you will be able to use to prove account ownership. We strongly encourage all our users to set up these proofs on all their email accounts. Proofs include:


  • Mobile phone number (to receive an SMS code)
  • Alternate email addresses
  • Trusted PCs
Prevention
Of course, the best way to fight hijacking is to prevent it from happening in the first place. The problem is that hijacking is fairly straightforward in many cases – it’s just a matter of getting your password. Hijackers get your password through several methods:


  • Guessing (although we made that a lot harder by banning common passwords)
  • Using phishing scams, in which the hijacker just gets you to give up your password
  • Installing key loggers and other malware on compromised computers
We’re fighting all of these not only in Hotmail, but in Windows and all the Windows Live services. For example, we’ve made IE more secure by detecting URLs with bad reputation, and we’ve added phishing and social engineering detection to SmartScreen™.

We’re not letting up

We’ve made tremendous progress in our battle against spam, but we know that spam and hijacking will continue to be a big problem for all service providers as long as there is economic incentive for the bad guys to do what they do. So we’re not letting up. We continue to invest in research and development to find ways to make it even harder for the spammers to get spam into your Inbox and to use Hotmail as a way of sending spam.

In my next post, I’ll go a bit deeper on one of the most insidious ways that spammers compromise your account: Phishing attacks. See you then.

Dick Craddock
Group Program Manager, Hotmail


aggbug.aspx

More...
 
Back
Top