Monitoring business segments

I think we have all heard the ‘best practice’ that speaks to monitoring portions of your network. The question is how do you do that even from a strictly network traffic view given the volume of traffic you are likely to see. I mean we can all talk ethereally about taking things piecemeal/not trying to boil the ocean, establishing baselines, monitor for anomalies because AV as a whole isn’t as trustworthy as we would like it to be from a detection standpoint. My question is how are people doing this?

In my old job I used ArcSight to create an approach that while worked and addressed a number of areas I’m not sure it was the best. Perhaps sharing this will spark some discussion or if nothing else help someone else get a step farther down the road if they are trying to do the same thing.

The over arching idea was to create a line graph with two lines that represented expected and unexpected traffic – note the dummied up graph here. In addition I created a table based report that was grouped by the following : source IP, dest IP, dest port, action (denied, accepted, etc), and then a count of events for those items. Depending on your tool and resources you might add fields like what device generated the logs, any asset information associated with the IPs etc. The last piece of the table report is to somehow show which traffic is expected or yet to be categorized. The timeframe of the reports are dependent upon a number of factors. I liked showing the graph for a 14day period and the table cover however often the overall report was going to be viewed: 24hrs for a daily review, weekly, etc.

In my opinion the graph provides a number of benefits:
• Quickly able to visually parse a large amount of data – no real spikes; on to the next report while you drink your coffee
• Ability to 1) see unexpected traffic and 2) see large spikes (or dips) in unexpected traffic
• It allows me to still see expected traffic on one report – I expect to see asset A talking to asset B but why did the volume triple last night? This is vs the tendency to remove expected traffic off the report.
• Quickly see ‘baseline’ traffic volumes (note above) – this is one reason I liked showing the graph of 14 days. The last 24hrs shows a spike; how does that compare to the same day last week? Backups are an example perhaps

Now I’m not saying this is THE way to do things, just a way that worked for us within a few different parameters. You could get to a point where the expected vs uncategorized traffic become so divergent that it is pointless to show the uncategorized traffic because its overall volume is so low. If so – congratulations!

For those with ArcSight I created this report by using a Trend that ran every hour (grouping results to reduce space). Because the line graph labels pull directly from values in a column I used a variable to populate ‘expected’ and ‘uncategorized.’ There is a trick to this however. Because as of 4.5 you can’t create a conditional variable against a trend that calls back to a filter….ahem…

/soapbox
This drives..well drove…me UP THE WALL!!!
/soapbox

I used a conditional variable on the query that populated the Trend itself. This way I just have to update the filter to include new sources of expected traffic.

How are folks tackling this issue?