Hi, my name is John Scarrow and, in conjunction with other product groups, I oversee service abuse and security issues for Windows Live and other Microsoft products such as Internet Explorer. Building from Dick Craddock’s previous post, I’m going to get into the nuts and bolts of how we fight phishing scams at Hotmail using SmartScreen® technology.
The chart below shows the trends for phishing attempts identified using SmartScreen in Internet Explorer 8 last year.
A typical phishing attack
Here’s an example of a phishing attempt that was caught by Hotmail in an attempt to steal the user’s Windows Live ID credentials.
Notice in this example that the e-mail appears to come from a valid domain ([email protected]), and includes text and images that look like e-mail sent from Microsoft. If you click the link in the e-mail, you get to the following (fake) Windows Live sign-in page:
This page, like the e-mail before it, uses a URL that may seem at first glance to be valid, and copies the images and text from the actual Windows Live sign-in page. Many customers, upon seeing this page, would simply type their credentials, which would be captured on the attacker’s website.
Once the perpetrator has the user’s credentials, they sign in to the victim’s account and send spam via Windows Live Hotmail to all the contacts in the victim’s address book. This form of social engineering spam is very effective as the spam appears to be coming from someone you know and trust.
To learn more about how to avoid phishing attacks, check out this article from Microsoft Online Safety.
In order to prevent these attacks from succeeding, Hotmail employs three key tactics to thwart phishing:
E-mail authentication
In the example above, the e-mail appeared to come from windowslivesupport.com, when in fact, it came from a completely different domain and service. This is a common tactic used by spammers and phishers known as domain spoofing. From a spam perspective, a higher perceived reputation of the sender can increase the percentage of people who open the message. From a phisher’s standpoint, it’s even more valuable, because recipients are also more likely to comply with the request to provide personal information such as a username and password.
There are two primary authentication technologies that are considered by most large scale e-mail providers to prevent domain spoofing: DomainKeys Identified Mail (DKIM) and Sender Policy Framework / SenderID (SPF). (There are notable differences and debates between SPF and SenderID, debates that go beyond the scope of this post. For the most part, Hotmail treats these two technologies the same way, so for simplicity, I’ll refer to both here simply as SenderID.) Hotmail currently supports only SenderID. However, we will be validating DKIM under certain circumstance in our next release.
SenderID is easier for legitimate commercial senders to adopt than DKIM, as it requires no new code deployment on the outbound mail servers. As noted by Online Trust Alliance, over 50% of e-mail sent by key sectors (those currently at the most risk of phishing attacks), use some form of SenderID. Although the adoption rate for DKIM has been slower to follow, many companies in these key sectors are now signing their e-mail with DKIM as well.
In the case of SenderID, commercial e-mail senders must identify all their outbound MTAs (Mail Transfer Agents or simply put - mail servers), collect and track the list of IP addresses they send e-mail from, and add all of this info to the TXT record in the DNS entry for their domain. For DKIM, they must deploy e-mail servers that support DKIM signing, and manage the key in the DNS server for access by e-mail receivers. Because of these requirements, it has taken e-mail senders time to fully support either DKIM or SenderID.
Using both DKIM and SenderID can work extremely well when all the pieces are in place. The requirements to support SenderID seem very simple, however, identifying all the mail servers in your organization can be challenging. Many IT departments are either decentralized, use multiple 3rd parties for outbound e-mail and marketing, or simply don’t have a good understanding of how to properly form their DNS TXT records. If the TXT record in DNS does not identify 100% of the servers that are authorized to send e-mail on behalf of the domain, many valid e-mail messages could inadvertently fail authentication and thus be incorrectly deleted by Hotmail and other mail providers. This creates an interesting challenge for Hotmail on the receiving end. When exactly can we delete mail that fails authentication, without accidentally deleting some good mail?
Which came first: the chicken or the egg?
Providers who are filtering incoming e-mail may stop short of deleting e-mail that fails authentication due to potentially incomplete records. But since strict enforcement of spoofed mail isn’t happening at most e-mail services, senders don’t have the motivation to adopt SenderID and DKIM standards. This creates a classic “chicken or egg” problem – not enough senders are authenticated properly so e-mail services can’t rely on authentication which means senders have no incentive to authenticate.
But this doesn’t have to be a chicken or egg problem, and Hotmail has come up with a unique solution. From the previous post you know that Hotmail has a reputation system in place that lets us evaluate e-mail coming from any specific IP address. We use this information to infer if a particular domain has a complete list of IP addresses of all their sending servers in their DNS TXT record. We do this by identifying messages from IP addresses that have a good reputation yet still fail authentication. In cases where very few good e-mail messages fail authentication, we know if a particular domain has done a good job identifying all their sending servers and have complete TXT records. This allows senders that do the right thing to get the benefits (the deletion of spoofed mail) without penalizing those that are still working their way up to 100% adoption.
Both DKIM and SenderID can generate false positives (mistakes) above and beyond failures in implementation as described above. For example we find that 1-3% of mail that fails SenderID fails due to mail forwarding services. Mail senders who are under heavy phishing attacks have repeatedly asked us to delete all these e-mail messages just to be sure, even though they know it could result in deleting good e-mail. We have always been concerned about this, as more and more folks are forwarding their other e-mail inboxes to Hotmail to take advantage of our filtering and user interface. We must insure that their valid e-mail makes the trip, forwarded or not.
At the last MAAWG (Messaging Anti-Abuse Working Group) and OTA (Online Trust Alliance) conferences in Philadelphia we announced our intention to verify DKIM authentication on inbound mail when SenderID authentication fails. This “Double Fail”, as it’s been termed, virtually eliminates the false positives that can result from either DKIM or SenderID alone. This allows Hotmail to confidently delete messages that fail both forms of authentication, when they come from senders who have complete records.
Finally, in the event that phishing mail does get into your inbox, we disable links by default for senders who are not a) on your contact list, b) marked as safe, or c) proven reliable by participating in the Sender Score Certified program. This means that before we enable the links we ask if you trust the sender, and allow you to decide whether you think that sender is safe. More information on mail treatment can be found on the Windows Live postmaster site.
E-mail authentication can be a very powerful tool to combat phishing. However, with all the inherent challenges, it requires creative solutions to make it work. At Hotmail we use authentication in conjunction with URL reputation and content filtering, and as a result, we are able to have a huge impact on phishing scams. In my next post I’ll cover our SmartScreen URL reputation system and how we use it across multiple products such as Internet Explorer and others. I’m looking forward to your feedback on this post (did I include too much technical detail, or do you want more?), as your comments will influence how I take on the next subject.
Thanks -
John Scarrow
General Manager Safety Services
More...
Phishing, as defined in Wikipedia, is the criminally fraudulent process of attempting to acquire sensitive information such as usernames, passwords and credit card details by masquerading as a trustworthy entity in an electronic communication.
Traditionally, the majority of phishing attacks target financial institutions and online retailers, and are aimed at acquiring log-in credentials that can be sold or used to fraudulently separate users from their money. There is a more recent trend now to phish for credentials for online services and social networking sites. Both use similar methods to phish for information. APWG (Anti-Phishing Working Group) reports that phishing numbers are not trending down.
The chart below shows the trends for phishing attempts identified using SmartScreen in Internet Explorer 8 last year.
A typical phishing attack
Here’s an example of a phishing attempt that was caught by Hotmail in an attempt to steal the user’s Windows Live ID credentials.
Notice in this example that the e-mail appears to come from a valid domain ([email protected]), and includes text and images that look like e-mail sent from Microsoft. If you click the link in the e-mail, you get to the following (fake) Windows Live sign-in page:
This page, like the e-mail before it, uses a URL that may seem at first glance to be valid, and copies the images and text from the actual Windows Live sign-in page. Many customers, upon seeing this page, would simply type their credentials, which would be captured on the attacker’s website.
Once the perpetrator has the user’s credentials, they sign in to the victim’s account and send spam via Windows Live Hotmail to all the contacts in the victim’s address book. This form of social engineering spam is very effective as the spam appears to be coming from someone you know and trust.
To learn more about how to avoid phishing attacks, check out this article from Microsoft Online Safety.
In order to prevent these attacks from succeeding, Hotmail employs three key tactics to thwart phishing:
- E-mail authentication looks at the sender e-mail address to make sure it is has not been spoofed. This prevents senders from pretending to send mail from another domain, for example mybank.com or mysite.com, and instead the sender must prove that they are who they say they are.
- Content filtering looks at the content of the e-mail to detect likely phishing attacks.
- URL or IP reputation looks at the reputation of links contained in the message and their domains, and identifies sites that are likely to host phishing content.
E-mail authentication
In the example above, the e-mail appeared to come from windowslivesupport.com, when in fact, it came from a completely different domain and service. This is a common tactic used by spammers and phishers known as domain spoofing. From a spam perspective, a higher perceived reputation of the sender can increase the percentage of people who open the message. From a phisher’s standpoint, it’s even more valuable, because recipients are also more likely to comply with the request to provide personal information such as a username and password.
There are two primary authentication technologies that are considered by most large scale e-mail providers to prevent domain spoofing: DomainKeys Identified Mail (DKIM) and Sender Policy Framework / SenderID (SPF). (There are notable differences and debates between SPF and SenderID, debates that go beyond the scope of this post. For the most part, Hotmail treats these two technologies the same way, so for simplicity, I’ll refer to both here simply as SenderID.) Hotmail currently supports only SenderID. However, we will be validating DKIM under certain circumstance in our next release.
SenderID is easier for legitimate commercial senders to adopt than DKIM, as it requires no new code deployment on the outbound mail servers. As noted by Online Trust Alliance, over 50% of e-mail sent by key sectors (those currently at the most risk of phishing attacks), use some form of SenderID. Although the adoption rate for DKIM has been slower to follow, many companies in these key sectors are now signing their e-mail with DKIM as well.
In the case of SenderID, commercial e-mail senders must identify all their outbound MTAs (Mail Transfer Agents or simply put - mail servers), collect and track the list of IP addresses they send e-mail from, and add all of this info to the TXT record in the DNS entry for their domain. For DKIM, they must deploy e-mail servers that support DKIM signing, and manage the key in the DNS server for access by e-mail receivers. Because of these requirements, it has taken e-mail senders time to fully support either DKIM or SenderID.
Using both DKIM and SenderID can work extremely well when all the pieces are in place. The requirements to support SenderID seem very simple, however, identifying all the mail servers in your organization can be challenging. Many IT departments are either decentralized, use multiple 3rd parties for outbound e-mail and marketing, or simply don’t have a good understanding of how to properly form their DNS TXT records. If the TXT record in DNS does not identify 100% of the servers that are authorized to send e-mail on behalf of the domain, many valid e-mail messages could inadvertently fail authentication and thus be incorrectly deleted by Hotmail and other mail providers. This creates an interesting challenge for Hotmail on the receiving end. When exactly can we delete mail that fails authentication, without accidentally deleting some good mail?
Which came first: the chicken or the egg?
Providers who are filtering incoming e-mail may stop short of deleting e-mail that fails authentication due to potentially incomplete records. But since strict enforcement of spoofed mail isn’t happening at most e-mail services, senders don’t have the motivation to adopt SenderID and DKIM standards. This creates a classic “chicken or egg” problem – not enough senders are authenticated properly so e-mail services can’t rely on authentication which means senders have no incentive to authenticate.
But this doesn’t have to be a chicken or egg problem, and Hotmail has come up with a unique solution. From the previous post you know that Hotmail has a reputation system in place that lets us evaluate e-mail coming from any specific IP address. We use this information to infer if a particular domain has a complete list of IP addresses of all their sending servers in their DNS TXT record. We do this by identifying messages from IP addresses that have a good reputation yet still fail authentication. In cases where very few good e-mail messages fail authentication, we know if a particular domain has done a good job identifying all their sending servers and have complete TXT records. This allows senders that do the right thing to get the benefits (the deletion of spoofed mail) without penalizing those that are still working their way up to 100% adoption.
Both DKIM and SenderID can generate false positives (mistakes) above and beyond failures in implementation as described above. For example we find that 1-3% of mail that fails SenderID fails due to mail forwarding services. Mail senders who are under heavy phishing attacks have repeatedly asked us to delete all these e-mail messages just to be sure, even though they know it could result in deleting good e-mail. We have always been concerned about this, as more and more folks are forwarding their other e-mail inboxes to Hotmail to take advantage of our filtering and user interface. We must insure that their valid e-mail makes the trip, forwarded or not.
At the last MAAWG (Messaging Anti-Abuse Working Group) and OTA (Online Trust Alliance) conferences in Philadelphia we announced our intention to verify DKIM authentication on inbound mail when SenderID authentication fails. This “Double Fail”, as it’s been termed, virtually eliminates the false positives that can result from either DKIM or SenderID alone. This allows Hotmail to confidently delete messages that fail both forms of authentication, when they come from senders who have complete records.
Finally, in the event that phishing mail does get into your inbox, we disable links by default for senders who are not a) on your contact list, b) marked as safe, or c) proven reliable by participating in the Sender Score Certified program. This means that before we enable the links we ask if you trust the sender, and allow you to decide whether you think that sender is safe. More information on mail treatment can be found on the Windows Live postmaster site.
E-mail authentication can be a very powerful tool to combat phishing. However, with all the inherent challenges, it requires creative solutions to make it work. At Hotmail we use authentication in conjunction with URL reputation and content filtering, and as a result, we are able to have a huge impact on phishing scams. In my next post I’ll cover our SmartScreen URL reputation system and how we use it across multiple products such as Internet Explorer and others. I’m looking forward to your feedback on this post (did I include too much technical detail, or do you want more?), as your comments will influence how I take on the next subject.
Thanks -
John Scarrow
General Manager Safety Services
More...