"(?i)response=(?.+)$" ----- - you should use < and > around the variable - (?<i>) - (?<i>) needs to come where it will appear on the event (ie, after the "response=") and i created few sample events ending with "response=digits" event 1 - Extract "from" and "to" fields using regular expressions. response=101 event 2 - If a raw event contains "From: Susan To: Bob", then from=Susan and to=Bob, response=404 event 3 - source="tutorialdata.zip:./www1/access.log" response=500 and this query picks up the response codes fine. sourcetype=responseREX | rex field=_raw "response=(?<i>.*)" | table _raw, i regarding the end of line $, these below two works same - response=(?.) and response=(?.)$ ... View more

looks like source and sourcetype are audittrail. i hope this is same for LDAP and local users as well. please check it and update us(for those who uses LDAP only) index=_audit source = audittrail sourcetype = audittrail ... View more

Please check this add-on... File/Directory Information Input Add-on The File Info add-on provides an input for indexing file meta-data (size, last modified dates, etc.) and indexing it in your Splunk instance to make it searchable. https://splunkbase.splunk.com/app/2776/ Or, you might need to do scripted input. Or, maybe, check this idea - - finding the size of all events from the log and adding it. on that Splunk Tutorial data, we have a file tutorialdata.zip:./www1/access.log and its size is 4.06 MB (4,262,086 bytes). thru this query, splunk shows its size as "4248458" (very close value, i think ;). approximately, i hope, its a good process) source="tutorialdata.zip:./www1/access.log" | eval esize=len(_raw) | stats sum(esize) by sourcetype ... View more

$SPLUNK_HOME/etc/system/default/alert_actions.conf OR $SPLUNK_HOME/etc/system/local/alert_actions.conf in this file, do you have any value for "ttl"?!? $SPLUNK_HOME/etc/system/default/savedsearches.conf OR $SPLUNK_HOME/etc/system/local/savedsearches.conf got any values for dispatch.ttl? ... View more

the question is bit not clear. may we know, some more info please.. 1. from this above sample log msg, do you want to query this event ah? Start of the event CPF1124 00 INFO Job 252582/QTMHHTTP/QIWIRELESS started on 07/04/16 at 00:00:21 in subsystem QHTTPSVR in QHTTPSVR. Job ent QIWIRELESS QTMHHTTP 252582 QWTPIIPP 0000 07/04/16 00:00:22.002029 QTMHHTTP <<<< End of the Event 2. the above sample log is a single event or multiple events? 3. from this sample log, can you tell us, how to find out the "start of the event" and "end of the event".. ... View more

for this task, i hope using metadata would be efficient. This one will find the hosts that have not sent any events for more than one day. | metadata type=hosts index=windows host=one (or .....)| where relative_time(now(), "-1d") > lastTime | convert ctime(lastTime) as Latest_Time | sort -lastTime | table host,Latest_Time A general query | metadata type=hosts | sort recentTime desc | convert ctime(recentTime) as Recent_Time ... View more

splunk document link for rex learning http://docs.splunk.com/Documentation/Splunk/6.4.2/Knowledge/AboutSplunkregularexpressions http://docs.splunk.com/Documentation/Splunk/6.4.2/SearchReference/Rex ... View more

edit - tried this and its working good. sourcetype=rexmemtotal | rex field=_raw "(MemTotal:\s+(?P<rexmemtotal>\d+))" | table rexmemtotal _raw ... View more

check this one.. index=main sourcetype=jmslogs | rex field=_raw "(?ms)^\s+(?P\d+\-\d+\-\d+\s+\d+:\d+:\d+,\d+) INFO \[(?P[^\]]+)" | table jmsContainer date _raw jmsContainer date _raw jmsContainer-2 2016-08-03 23:32:07,551 2016-08-03 23:32:07,551 INFO [jmsContainer-2] ... jmsContainer-15 2016-08-03 23:43:06,660 2016-08-03 23:43:06,660 INFO [jmsContainer-15] ... jmsContainer-1 2016-08-03 23:20:06,581 2016-08-03 23:20:06,581 INFO [jmsContainer-1] ... jmsContainer-10 2016-08-03 23:39:06,123 2016-08-03 23:39:06,123 INFO [jmsContainer-10] ... jmsContainer-1 2016-08-03 23:31:06,453 2016-08-03 23:31:06,453 INFO [jmsContainer-1] ... jmsContainer-5 2016-08-03 23:51:00,607 2016-08-03 23:51:00,607 INFO [jmsContainer-5] ... ... View more

please check this sourcetype events - f5:bigip:syslog if you have any sample events, we can create the splunk query. ... View more

hi, 10billion seems a very huge number. are you sure?.. also did you check the set diff configurations on the limits.conf file ah..please update us.. ... View more

oh ok. i thought cisco:asa logs may have same format. seems your environment is different. ok, thanks. ... View more

as i checked, "sourcetype=cisco:asa" events are not having a field "dev" tried it without "dev" and its working fine.. src=122.15.158.173 sourcetype=cisco:asa "Deny*"|stats max(_time) as last_time by host,sourcetype,action,dest,dest_ip,dest_port,index,msg,src,src_ip,src_port,vendor_action |dedup last_time,host,sourcetype,action,dest,dest_ip,dest_port,index,msg,src,src_ip,src_port,vendor_action ... View more

10 billion !!!... i think, you may need to edit the query timelines and do multiple queries. with default values for set command, it wont return 10 billion. http://docs.splunk.com/Documentation/Splunk/6.4.2/SearchReference/Set Output limitations There is a limit on the quantity of results that come out of the invoked subsearches that the set command receives to operate on. If this limit is exceeded, the input result set to the diff command is silently truncated. If you have Splunk Enterprise, you can adjust this limit by editing the limits.conf file and changing the maxout value in the subsearch stanza. If this value is altered, the default quantity of results coming from a variety of subsearch scenarios are altered. Note that very large values might cause extensive stalls during the 'parsing' phase of a search, which is when subsearches run. The default value for this limit is 10000. Result rows limitations By default the set command attempts to traverse a maximum of 50000 items from each subsearch. If the number of input results from either search exceeds this limit, the set command silently ignores the remaining events. By default, the maxout setting for subsearches prevents the number of results from exceeding this limit. If you have Splunk Enterprise, you can change this limit by editing the maxresultrows setting in the set stanza in the limits.conf file. ... View more

maybe, can you give us some example data, so that it will be easy to understand your issue.. ... View more

i assume you would want to configure "Discard specific events and keep the rest" type events routing. one example configuration i updated below.. for further reference on this topic- http://docs.splunk.com/Documentation/Splunk/6.4.2/Forwarding/Routeandfilterdatad Discard specific events and keep the rest This example discards all sshd events in /var/log/messages by sending them to nullQueue: In props.conf, set the TRANSFORMS-null attribute: [source::/var/log/messages] TRANSFORMS-null= setnull 2. Create a corresponding stanza in transforms.conf. Set DEST_KEY to "queue" and FORMAT to "nullQueue": [setnull] REGEX = [sshd] DEST_KEY = queue FORMAT = nullQueue 3. Restart Splunk Enterprise. ... View more

Check this one.. from a similar issue post - The only reliable source for license usage is the $SPLUNK_HOME/var/log/splunk/license_usage.log file on your license master instance, and unfortunately it does not split usage by index. There is, however, a sampled record of kilobytes indexed for the top 10 most active indexes every 30s in metrics.log. Of course if you have less than 10 active indexes, this is not an issue. So, for a report showing daily license usage over the past 1 day, you would run: index=_internal source=*license_usage.log type=RolloverSummary earliest=-1d |eval GB = b/1024/1024/1024 | eval _time = _time - 43200 | timechart span=1d sum(GB) AS "Total GB used" For a report showing estimated daily volume indexed (whether it counted against your license quota or not) over the past 7 days, you would run: index=_internal (host=indexer1 OR host=indexer2 OR host=license_master) source=*metrics.log group=per_index_thruput earliest=-7d | timechart span=1d sum(eval(kb/1024)) AS "MB indexed" by series ... View more

May I know if that rex on my reply worked fine or not. ... View more

4693489783100 bytes is 4693.4897831 GB. so, something wrong with "Bytes" calculation only. (earliest=-1d@d latest=@d index=_internal source=license_usage.log type=Usage | stats sum(b) AS Bytes by st) ... View more

nope. check_for_header is not deprecated. the latest version 6.4.2 document reference - http://docs.splunk.com/Documentation/Splunk/6.4.2/Admin/Propsconf on this page, search for CHECK_FOR_HEADER = [true|false] ... View more

i am not getting you. by "trial and error", you meant, you already tried this ah? the above one is from this post and six guys upvoted that. https://answers.splunk.com/answers/1041/how-to-ignore-the-titile-line-of-the-csv-file-in-the-result-and-display-in-a-kv-format.html so, i think this above info is a correct one. ... View more