Sunday, November 20, 2016

Find saved searches in Splunk that are failing

I hope to circle back to this eventually. Until then --- enjoy:

index=_internal log_level=ERROR SavedSplunker | stats count as Count by host message | rex field=message "savedsearch_id=\"(?<Author>[^;]+);(?<App>[^;]+);(?<Search>[^\"]+)\"(?:, message=)?(?<Message>.+)" | table host App Search Author Message Count | eventstats sum(Count) as total by host | eventstats sum(Count) as foo by host App | sort -total -foo -Count | fields - total foo

Saturday, April 9, 2016

Splunk admin tasks after you start getting data in...

I had the rather unique privilege to post a 3 part blog series on Splunk's official site recently. The focus was on some administration tasks Splunk admins should work into their routine. There is a level of assumption when users search in Splunk - these hosts are really these hosts and events that are observed within a time range really happened then. The series talks through a couple methodologies to validate those assumptions

  • Part 1 - Validating host field values: link
  • Part 2 - Validating agent host's system time: link
  • Part 3 - Getting a feel for data ingestion latency: link