Event Analysis Training – SSH Brute Forcing with Mixed Log Sources

I was recently working with a Log Correlation Engine customer who had gone through a typical deployment. Tenable advises customers to carefully consider which system and application logs they want their LCE Clients to send to the LCE server, in addition to a variety of Snort, Firewall and network activity logs. In this case, the customer had recently configured their system to send SSH logs to the LCE whereas before, they were only getting "network" events. When I had the opportunity to chat with them, they were very concerned about various worms or potential intruders performing a network scan such as those shown below:

For a period of time during the evening (the grey area is 6:00 PM to 6:00 AM), there were many SSH sessions detected by Snort, the Tenable Network Monitor (the events beginning with “TNM”) and by the systems where an actual login was attempted.

The customer was concerned because they had received multiple correlated “Password Guessing” events during the evening, but only on a few systems. Only a partial number of servers had SSH and full system logs configured to send to the LCE. However, Snort, the TNM and even the Passive Vulnerability Scanner were reporting evidence of attacks against the entire network. The SSH authentication failure and other SSH messages were all in system logs, not all of which were being sent to the LCE.

In this case, I told the customer there was very little to be concerned about and explained that the Log Correlation Engine correlation script that looks for password guessing also looks for a successful password guess. Visually inspecting the graph, there were no login events of any type recorded. The LCE normalized all valid login events from SSH. If this attacker had leveraged an SSH key from the Debian debacle or indeed guessed a password, a successful SSH login would have been reported.

I was concerned that some of the other systems without full logging could have been compromised. To investigate this possibility, I suggested that the customer perform an IP summary to get a list of all internal systems and then check each of the Tenable Network Monitor Session Completed events to look for any session times or bytes transferred values that looked different from the others. In doing this, the customer was able to conclude that the network sessions to all of the other servers were very similar. If they had had full system logs being sent to the LCE Server, this form of analysis would have been quicker.

Last, I was concerned that some of the network sessions could have included some sort of buffer overflow attack and that the password guessing was a decoy. The customer verified that all of their SSH daemons were of the same build, OS and patch level. If there had been a successful attack on the target system for which we did see logs, we didn’t see any odd behavior on that host. There certainly were no outbound connections originating from that host, or new ports browsing. Both of these types of activity would be logged by the Tenable Network Monitor or the Passive Vulnerability Scanner.

For More Information

Tenable has written several Event Analysis Training articles that cover many different aspects of event analysis such as blacklisted IP address correlation, different types of worm traffic patterns and botnets, including:

• Event Analysis Training -- An aggressive active worm analysis that isn’t Conficker
• Event Analysis Training -- Working with "BlackLists"
• Event Analysis Training -- Long Term Blacklist Activity
• Using "New Port Browsing" Events to find Worm/Trojan/Rootkit Activity