Thursday, August 12, 2010

Wrapping your arms around ArcSight Trends - Part 1

Anyone else’s drive into work getting a little darker? I feel like the summer has FLOWN by. Before too long it will almost be back to a military 0’dark 30 to 0’dark 30 - at least relative to the sun; I'm not working additional hours :)

There are a number of things rattling around in my head I’d like to write about and even some things I indicated I would write about in previous posts. Work, family, summer, and the cumulative lost sleep effects from fighting through some weird sinus issue for the last 6 months have conspired to work against this goal. 60 back pokes and 22 shots in the arm have at least indicated it not being allergies. Good times.

For S&Gs I think this will be the start of a short series of posts/articles/whatever on Trends. My next post may or may not be the continuation lol. If it isn’t I WILL come back to it.

Trends

What’s there to say other than they rock. We were lucky enough to have deployed ArcSight right after 4.0 came out so we had access to them from the beginning of our overall learning curve. Throw a Query Viewer on top of the Trend (ESM 4.5) and now they don’t just rock; they start to roll. I say start because there are a number of things I wish you could do with them still. The message we received though; and took to heart; was if you have reports you are generating on a regular basis, throw the data into a Trend and then base your report on it. Since 4.0 has been out for a while now I don’t really want to spend any time one the basics of how to create one (create query and point your Trend to it – whoops, I covered the basics) as much as a couple of methodologies to make sure your Trends are healthy. Well…maybe not “healthy” as much as some of the things I have done to help wrap our arms around the ones we have created.
As we started creating Trends we eventually ran into a few issues. I will cover some of the most common gotchas in a later post. One scenario we thought of was we had too many Trends starting at the same time. By default if you don’t change when your Trend polls then all of them will start on the hour (HH:00). One solution is to put your mouse over the minute selector on the Trend schedule tab, close your eyes, and click at variable speed trying to alternate between increasing or decreasing the start minute. Alternatively you could open the latest Anton Chuvakin blog post and click the minute selector for every word in the first sentence. While either method solves the issue of all your Trends starting on the hour it really doesn’t address the more global one of distributing your start times across the available 60 minutes.
I could be missing something but there doesn't seem to be an out of the box way to get a handle on when your Trends actually kick off. Opening each Trend is an option though I would suggest a slightly different approach. The solution for me was to create a report by using the deviceEvent field populated with /Trend/Run/Started. There is a specific eventID but I don’t recall it right now (guessing it might be trend:100). At any rate the initial report was endTime but with the minute aggregate function and then a count of unique fileName which holds the Trend's name. This gave me a count of the number of Trends starting in a particular minute. While that is a nice snapshot you might as well take the count off the fileName field allowing you to see which Trends are starting when...or keep both for a rainy day. With data in hand you now can adjust the start times as needed. If most of your Trends run hourly you really only need to have the report go back over the last hour which shouldn't take long at all.
 
This report gives you some decent data but ultimately doesn't contain everything I would like to see. More to follow. Would be interested in things you have done to wrap your arms around Trends.
 
Mark

No comments:

Post a Comment