Statistical vs Rule based Threat Detection

A number of different discussions have led me to think about the difference between log management and SIEM when it comes to their use and play in threat detection. Of the many items that could be discussed what I came to is the difference between statistical and rule based threat detection.

An oft used analogy, even referenced in the Verizon DBIR, is the difference between looking at haystacks and needles in haystacks. A statistical detection methodology might be to review the top N of X activity within Y timeframe. The point of this exercise is twofold. The first is simply to look at the “curve” of the numbers involved in the top 5 relative to the top 10 relative to the top 30 or just a spike in a line graph showing the volume of logs collected. All of which might indicate something has happened and might be worth diving deep on. The second is to help dial in your tools by whitelisting systems performing normal activity. If you are looking for outbound SMTP traffic sorted by volume in a day, you should be able to easily spot your email gateways. Whitelist them and the next time the report is run the top of the list might contain compromised systems. By and large many log management systems should be able to accomplish this sort of activity.

At some point though you will want to focus in on specific threat activity. Take today’s SANs Diary update on Run Forest Run or Sality, Tidserv, whatever. In this case you have specific information and want to receive an alert when one of your internal systems hits an IP, URL, or multi-step pattern of activity. This is your rule based needle finding capability. Generally speaking this requires a rule engine of varying level of sophistication located within some point solution or product. Statistical detection won’t really be able to get you this.

The challenge is many point solutions, by design or omission, aren’t able to factor in the larger view in reviewing the needles found. MSSPs are notorious (too strong?) for this but then so are things like IPS. “We saw this and this so we wrapped it up in a pretty bow for you” ….great, but I need more context. The SIEM technology space, in general, was supposed to fill this gap. Not only can you develop specific rules to find needles in the overall stream of log consciousness but to a greater or lesser extent, based on vendor/tool/administrator, use them in a more statistical way. I think a lot of “Mah SIEM sux” mentality comes from how you approach your SIEM relative to this overall issue. That is a rabbit hole that I don’t want to go down though.

Especially when you first start out, if you dive directly to a rules based approach you will have a harder time seeing the forest for the trees and depending on the tool used will be frustrated that you can’t move that lens backward. In other words dealing with individual infections IS key but if you are so focused on the individual detections you lose sight of the bigger picture you can do yourself a disservice. On the other hand if you just do a statistical approach and never grow you are going to miss the needles you need to find.

I would argue there is a direct correlation between shop maturity and the ability to full leverage rule based technology. If you are just starting out I suggest you will see more value with a log management, statistical threat detection methodology. This will allow you to get to know your data – as strange as that might sound – which will allow you to better dial in your rule based solutions.

SIEM alerts and an analogy to help people 'get it'

It may just be me but does anyone else struggle with people in general and over-arching 'management' not really getting it that when you get or see a security based event from your LM or SIEM; that this is just the beginning of a process and not the end? I mean they say they get it but you just have this feeling that they really don’t get it? Now I’m not saying my statement reflects my current management, it is more a generalized observation as you start to bring people into the conversation who might not have been exposed to this sort of technology or the exposure is limited to a conversation about needing to monitor something. And there is a level of irony as some of these folks are in the actual vendor space.

I’m sort of an analogy guy and this has been one of those things that has plagued me for some years now – what is a good equivalence that helps people get it? I thought of one today that might work but still feels sort of rough. Figured I would use this as a sounding board and hopefully get some feedback. Who knows, maybe just putting it down on paper will help refine it.

While driving around in your car you generally are vaguely aware of, but may or may not pay a whole lot of attention to, your gas gauge. Some probably have more of an alert mentality and wait for the gas to get to the last X% when you get the audible chime indicating you are low. You now are more focused on the situation and ‘remediate’ the issue by stopping off at a gas station and filing up. I think this is somewhat analogous what ‘management’ thinks of when they hear things along the line of monitoring alerts. That the process is generall closed and ends when you get the alert. Alert > gas station > fixed. The problem is that isn't what we are talking about. SIEM/LM alerts, I think, are more akin to all of the gauges in your car that don’t even show until there is an issue. Things like low oil pressure, over heating, your engine service light. Any of those things coming on means something isn’t right somewhere but unless you pop the hood and/or break out some diagnostic equipment you don't know the severity. Could be a complex, multi-component breakdown or just something a little out of whack that takes a simple fix. Oh and don't forget the number of things that can happen to your car that you don't have a gauge for: breaks, suspension, tire alignment, etc. Multiply that picture by the number of end point systems you have in your environment and I think we are getting a little closer to a decent analogy. The point is the alert often isn't actionable other than letting you know you need to start an investigative process.

What do you think - does it work? Do you have suggestions, tweaks, or better analogies that have worked for you as you try to convey to people that monitoring alerts is more than just sitting around waiting for them to pop in your inbox?

Is ArcSight harder to use?

Interestingly enough (if I read Google Analytics correctly) the post I created back at the beginning of November related to the question of is ArcSight hard to use was the most visited page over the last month. In fact I think it has been on or towards the top of the list since I posted it. Since I have some time on my hands I figured I would write a sort of follow up to that post. Once again I will state that my only SIEM experience is with ArcSight so maybe someone will speak up for the other SIEM engines out there and/or tell me if I’m way off base.

I still maintain asking if something is “hard”, generally speaking, often only gets you only so far. Having some context implied in your question might get you an answer closer to what you really are looking for. Is calculus hard? Depends on who you are talking to. Is calculus harder than basic algebra? Yup. Is “ArcSight” hard to use? Depends on who you are talking to. Is ArcSight harder to use than some other SIEM? Ahh now we are getting somewhere. I did have sort of an interesting thought while reading the SIEM implementation book I wrote a review of back in January (at least it was interesting to me).

A brief FUD and SIEM thought

A buddy at work ran across a Wall Street Journal article that mentioned "Arcsight’s profits were up over 300% last year despite the recession and sales were up over 30%." This was basically attributed to FUD.

Of the many things you could say about those stats the one that is most on my mind is that I hope the companies that bought the software didn't just buy it to "do SIEM" or check a box and deploy it in a manner that will simply lead to failure or under utilization. At the very least I hope they have or are willing to set aside most of an FTE for care and feeding of the system.

I had a conversation with an ArcSight VP who mentioned one of their observations/frustrations is going back to a company two years after their professional services helped with an install and basically seeing no new content. To that end I think there is a very real effort within the company to make things easier and introduce some scaling (downward) for medium and smaller customers. The reality is though, imho, SIEMs aren't an InfoSec product like other point specific InfoSec solutions. In other words you buy AV, IPS, FW, etc to address a specific issue in your environment. A SIEM is really more of a platform you can use to survey normalized event data from across your network. To that end it requires effort to build and maintain content as well as effort to (crazy thought here) respond to the issues found.

Monitoring logs and SIEMs

So there are several SIEM articles that tackle SIEMs vs Log Management, reasons you might want to use one in your environment, and what you might hope to see with one. I’m kinda a simple guy though and while I have recently tried to wrap my head around the nuances between monitoring log management style vs “just” alerting and how you might use a SIEM to both ends I was hit with a rather simplistic realization - the SIEM is already monitoring the logs.

See, I said it was simplistic. Here is what I mean though. I currently have multiple tickets open with ArcSight support and having reviewed multiple sets of manager logs one of the techs was kind enough to open an additional ticket based on some things she saw. I had three takeaways:
1. I should monitor the ArcSight logs. (I’m insightful aren’t I)
2. If I had been monitoring them I would have picked up on the warnings/errors/whatever that was seen
3. My manager logs rolled every 2 hours – used to anyway.

This, to me, makes the manager logs a perfect candidate for insertion into the SIEM. Granted part of the “perfect” is that the logs are being generated by something designed to suck in logs so it doesn’t seem to make a whole lot of sense to have something other than itself actually monitor…itself. The other side of this is while there is value in actually looking at the logs periodically I think this is also a perfect case of being able to just set up alerts.

The real takeaway then is maybe an adjustment to the argument that SIEMs are an “analyst in a box.” While they aren’t really an analyst they are at least some/part/multiple/an FTE who is, or at least capable of, monitoring logs. The next takeaway is according to the 2009 Verizon Data Breach Investigations Report we should be looking at application logs anyway. I mean the OS logs I am pulling in from the server hosting the ESM certainly aren’t telling me anything about these “guardoverflow” events that I should apparently search for in my logs in order to fix some data monitors.

Of SIEMs and Gymnastics

In thinking about SIEMs overall relative to a thought that has been building over the last month or so I had a flashback to my cheerleading days.

Don’t laugh. For a slightly introverted guy, cheerleading at a college with a girl to guy ratio of 7:1 = win.

One of the differences between a back hand spring and a back tuck is the general ability to transfer momentum from one movement into the next. This is a lot easier with a back hand spring since the motion is front to back vs the tuck’s up and down. It is one of the main reasons why you are much more apt to see a series end in a back tuck vs start with one and transition to other things.

My problem is I learned how to do a back tuck – the finishing move – first. This translated into me not really being able to do multiple back hand springs in a series.

So, how does this relate?

If you look at the maturity models from the last post, SIEMs are toward the right side. IMO then and based on the language used by the vendors I have talked with you have use cases all about alerting on X in your environment. SIEMs are (should be?) the finishing move back tucks of the InfoSec world. Based on your environment though how are you using it? Starting at alerting and backing into monitoring? Moving from monitoring into alterting? Can your SIEM do both or maybe you need just one or the other? I really think the difference between monitoring and alerting is more than just semantics.

I’m the newb here - am I off my rocker?