Thanks, you just helped a noob from losing his mind! ... View more

I think you're pretty far off here. eval delta=_time/3 says to take the _time value, which will be the number of seconds since 1970 (ie 1320916082), and divide it by 3. So those 'delta' values will look like "440305360.667". Then the join command will theoretically combine the events from each set that happen to have the exact same values for this field, which will only happen when there are events on each side that happened in the identical second. Backing up a second, if you're trying to find instances where these two kinds of events occur within N seconds of eachother you might consider matching them with a simple OR command in the search, index=tbb sourcetype=tbb_server ("No UserSession exists with the primary key" OR logout) and then piping it to the transaction command, using its startswith , endswith and maybe maxspan arguments to group the raw events into little transactions and then after that doing something like | search duration<3 . http://docs.splunk.com/Documentation/Splunk/latest/SearchReference/transaction ... View more

You need the map command: index=os sourcetype=vmstat host="tbbpapp1.xyz.com" | multikv | stats last(_time) as starttime| eval endtime=starttime+10 | map search="search earliest=$starttime$ latest=$endtime$ index=tbb sourcetype=\"access_combined\"|rex \"10\.\d+\.\d+\.\d+\s(?<source_ip>[\d+\.+]+)\s\[\d+\"" ... View more

Thank you. That fixed it. 🙂 ... View more

thank you it worked. |transaction _time|rex "(?s)Query_time:\s+(?P \d+.\d+)\s+Lock_time.?SET\stimestamp=\d+;\n(?P . ?);"|table masterdb_query_time masterdb_query|sort -masterdb_query_time|head 10 ... View more

Okay, I see one problem with this search: you define a field named "url" in the rex command, but you use it as "URL" in the timechart command. Field names are case-sensitive. Assuming that your regular expression is correct in the rex command: index=xyz sourcetype=xyz GET |rex ="GET|POSTs+(?P<url>[^ ]+?)s+HTTP"|timechart span=10m count by url useother=false usenull=false should work. Note that I also changed count(URL) to just count You do not need to count(url) if all events have the url field. ... View more

responding late since i ran into this and was left scratching my help. so hopefully this helps someone else. you have to very careful with the source data format (what is being logged in the log files by the web server). in my case, i was using a CustomLog configuration in Apache, and using an unused field (%l - Remote logname), with some request header value but was double quoting the value. splunk wasn't quite liking that, once i removed the double quotes, splunk loved it. ... View more

Depending on the data you are searching for you could also use the <searchPostProcess> tag which allows you to use a PostProcess search. There are examples in the UI Example app that you can download off of Splunkbase. This would allow you to conduct one search over that timeframe and let the other searches use its results. ... View more

The previous answers are right, but I'd like to point out that searching with a leading wildcard is much less efficient than having a wildcard on the suffix. In other words, looking for "Blah*" is pretty quick because splunk can do an efficient lookup to say find terms start with "Blah". Whereas, searching for "*Blah", splunk must scan all terms looking for ones that ends with "Blah". This type of index lookup will always take longer, but you may or may not notice; that's going to depend on how many unique terms your index contains. So my suggestion would be to build a list of all possible exceptions types and put them into a big "OR" list: Step 1: Figure out how may different "*Exception" patterns you really have in your data. (you may want to search over a long time period to make sure you don't miss any.) source="/home/xyz.log" *Exception: | regex "\.(?<exception>\w+Exception:)" | dedup exception Step 2: Take that list of terms and combine them into your original search, something like this: source="/home/xyz.log" (ParseException: OR UserException: OR BlahException: OR ...) Assuming you don't have all that many exception types, you should end up with a faster search. You'll also have to ask yourself: How often do new exception types show up? Which is preferable? (1) good performance with the possibly of missing events when new exception types show up, or (2) never missing events, but having a slower search. There's a helpful video about segmentation here: http://www.splunk.com/view/SP-CAAACXB http://www.splunk.com/web_assets/video/2008/dev/PreviewPeeks/Sorkin_Segmentation.swf ... View more