So how big ARE Windows Logs?

In my last post I mentioned how I was re-writing a few Windows events to cut down on Splunk license issues. In trying to size log management solutions in the past I've looked for lists or rules of thumb when it comes to the size of Windows events but never really found anything. That being the case hopefully someone will find this useful. I ran a query just now in Splunk to get the average byte count per Windows event ID. If you need to figure out log management license sorts of things this could give you a ROM by which to multiply a sampling of your event count against (as in number of logs on one server over 24hrs * number of related servers). After the cut you will find a 'csv' listing the Event Viewer (sourcetype), Event ID (EventCode), and average bytes for that ID. Enjoy. Oh - the average byte count for all of our Windows logs is 630.

Taming verbose Windows logs in Splunk

As you get into the world of logs you quickly realize how 'heavy' Windows logs are. By that I mean verbose. In this space verbose = length and log length translates to increased storage and licensing issues. Many log generators simply say this did that or this talked to that over this port. Pretty quick and dirty. Windows logs are generally along the lines of "dear reader, I've observed many events in the course of my life and here is something I thought I should bring to your attention. I will go on at length about this though really only give small pieces of important information with little to no explanation forcing you to scour the Internet looking for others who have gone through this self same issue." I ran a quick search in my Splunk environment and found the average Windows event code to be 630 bytes.

Splunk User Activity - Apps and Dashboards

At some level time on this blog can be marked by when I'm having a car worked on. Today isn't an exception. While that means the overall frequency is low the upshot is I get to spend my morning working in Chick-fil-A with some good breakfast in mah belly and good music in the background.

A couple days ago I wanted to get a feel for where users were going/how they were using Splunk. This is one of those things that seems easy but the further you go down this path the more offshoots there are. At any rate I was wanting to focus on UI navigation so knew I was going to be looking at the web access logs vs searches performed. There is a couple screens in SoS but they weren't working. There are also a couple screens in the search app (in 6x that is in Activity > System Activity) but those views, while working, didn't display what I was really after.

In looking at the queries behind all of these screens I ended up with the following:

index=_internal source=*web_access.log* uri=*/app/* uri!=*/images/* status=200 | rex field=uri_path "app/(?<app>[^/]+)/(?<view>.+)" | where app!="launcher" AND view!="Home" AND view!="home" AND view!="landing" | table _time host app view user _raw