In the mean time why don't you try appending "+0000" to your REPORTED_DATE and convert to epoch including the timezone specifier This was  perfect and worked great. I am very interested in macros I have never done them can you help me build the ones out you did        ... View more

template (name="trendmicro" type="string" string="/PROD/trendmicro/%fromhost-ip%/trendmicro.log") template (name="asa" type="string" string="/PROD/asa/%fromhost-ip%/asa.log")   ruleset(name="remote-udp"){ if $fromhost-ip == '10.100.' then { action(type="omfile" dynafile="trendmicro") } if $fromhost-ip == '10.0.4' then { action(type="omfile" dynafile="trendmicro") } if $fromhost-ip == '10.135.' or $fromhost-ip == '10.0.' then { action(type="omfile" dynafile="asa") } if $fromhost-ip=='10.19' or $fromhost-ip == '10.19' then { action(type="omfile" dynafile="fireeye") } stop }   # bind ruleset to tcp listener and activate it: input(type="imudp" port="514" ruleset="remote-udp")     template (name="rsyslog-fmt" type="string" string="%TIMESTAMP% %HOSTNAME% %syslogtag%%msg:::sp-if-no-1st-sp%%msg:::drop-last-lf%\n" ) template (name="Checkpoint" type="string" string="/var/log/splunk/Checkpoint/%HOSTNAME%/checkpoint.log") template (name="Checkpoint_sys" type="string" string="/var/log/splunk/Checkpoint_sys/%HOSTNAME%/checkpoint.log") template (name="F5" type="string" string="/var/log/splunk/F5/%HOSTNAME%/f5_waf.log") ruleset (name="network-logs") { #if $HOSTNAME startswith "SCMD-SPL-DEPS" then { action (type="omfile" dynafile="test" template="rsyslog-fmt") stop } #if $fromhost-ip=="10.40.71" then { action (type="omfile" dynafile="test" template="rsyslog-fmt") stop } action (type="omfile" file="/var/log/splunk/uncategorised.log" template="rsyslog-fmt-unc") stop } #input (type="imtcp" port="514" ruleset="network-logs") ... View more

I'll do my best to make pseudo SPL here:) index="prod" source="mysource" | transaction startswith=(fieldname=start_field_value OR filename=alternate_start_field_value) endswith=(diff_fieldname=end_value OR diff_fieldname2=alternate_fieldvalue) |stats count You will need to encapsulate your search in "()". If you could share field names, values desired, that would help. I used  an old eventgen data set and came up with the following. You can use this with your index(es), sourcetypes, etc.  index=main | transaction clientip JSESSIONID startswith=(status=200 action=addtocart) endswith=(status=200 AND action=purchase) | stats count by host You can insert SPL into the "()" to define start and end. I would avoid using clear text search for any of the start/end filters though. ... View more

Your base search will be specific to your environment (i.e. Windows Event codes, ssh login messages, etc), but if you get your base search, you can than do something like: "yoursearch" | chart count over user by host Again, it will depend on your source data. If you give a bit more info (sourcetypes, etc), be happy to respond. ... View more

Yours worked in 99% of the events. What I finally ended up with was a really long regex. BREAK_ONLY_BEFORE = \w{3}\s\d{1,2}\s\d{1,2}\:\d{1,2}\:\d{1,2}\s\d{1,3}\.\d{1,3}\.\d{1,3}\.\d{1,3}\s\[ ... View more

You are restarting Splunk after making props.conf changes, right? Also, the changes only apply to newly-indexed events. What struggles are you having with the timestamp configuration? ... View more

My error was in the LINE_BREAKER. I had inadvertently escaped some [ characters with \ when I really wanted them to serve as alternate character specifications. So the bad version was: LINE_BREAKER = ([\r\n]+)([?\d{4}-\d{2}-\d{2}\s\d{2}:\d{2}:\d{2}\,\d{3}]?\s|\d{4}-\d{2}-\d{2}\s\d{2}:\d{2}:\d{2}\s|#{0,4}<\w{3}\s\d{1,2}\,\s\d{4}\s\d{1,2}:\d{2}:\d{2}\s[AP]M\s\w{3,}>\s|\w{3}\s\d{1,2}\,\s\d{4}\s\d{1,2}:\d{2}:\d{2}\s[AP]M\s|\d{2}:\d{2}:\d{2}[.,]\d{3}\s|[DEBUG]\s\d{8}\s\d{2}:\d{2}:\d{2}[.,]\d{3}\s|[INFO]\s\d{8}\s\d{2}:\d{2}:\d{2}[.,]\d{3}\s|[ERROR]\s\d{8}\s\d{2}:\d{2}:\d{2}[.,]\d{3}\s|[WARN]\s\d{8}\s\d{2}:\d{2}:\d{2}[.,]\d{3}\s) The corrected version is: LINE_BREAKER = ([\r\n]+)([?\d{4}-\d{2}-\d{2}\s\d{2}:\d{2}:\d{2}\,\d{3}]?\s|\d{4}-\d{2}-\d{2}\s\d{2}:\d{2}:\d{2}\s|#{0,4}<\w{3}\s\d{1,2}\,\s\d{4}\s\d{1,2}:\d{2}:\d{2}\s[AP]M\s\w{3,}>\s|\w{3}\s\d{1,2}\,\s\d{4}\s\d{1,2}:\d{2}:\d{2}\s[AP]M\s|\d{2}:\d{2}:\d{2}[.,]\d{3}\s|[DEBUG]\s\d{8}\s\d{2}:\d{2}:\d{2}[.,]\d{3}\s|[INFO]\s\d{8}\s\d{2}:\d{2}:\d{2}[.,]\d{3}\s|[ERROR]\s\d{8}\s\d{2}:\d{2}:\d{2}[.,]\d{3}\s|[WARN]\s\d{8}\s\d{2}:\d{2}:\d{2}[.,]\d{3}\s) This regex is necessarily complex due to the variety of date/time formats I found were present in my weblogic_stdout events. ... View more

Hi ejwade, You can send specific events to the nullQueue to discard them at the indexer/heavy forwarder level. In your case it would look like: props.conf [source::<bro_logs_source>] TRANSFORMS-null= set null transfroms.conf [setnull] REGEX = <your_regex> (for you something that deals with internal A record) DEST_KEY = queue FORMAT = nullQueue You can have a read through the "Filter event data and send to queues" section at http://docs.splunk.com/Documentation/SplunkCloud/6.6.3/Forwarding/Routeandfilterdatad. Hope that's helpful! ... View more

Yes, I updated my answer along similar lines with a great deal of instructional detail on how to use such SPL. ... View more

I'd do the following in props.conf: TIME_FORMAT=%m/%d/%y","%H:%M:%S TIME_PREFIX=\d+"," BREAK_ONLY_BEFORE_DATE=true Based on your sample, that should break events correctly. If not, can u paste more sample events? ... View more

For "Portal_eventlog" sourcetype, I am seeing only the standards fields such as host, index, source, sourcetype, linecount, splunk_server. The log file is an XML. So, I am expecting each tag as a fields. Note: Some people mentioned that we have to have props & transform.config in the forwarder too. I have tried that too & didn't work. ... View more

@cmerriman - a day... a week...Hey, to God it's a thousand years anyway, right? I also notice that what the OP posted, his "yesterday" search ended at the hour cutoff beforen 24 secondws ago. ... View more

For future reference - FORMAT = $1::$2 (where the REGEX extracts both the field name and the field value) from Create custom fields at index time ... View more

This worked, thank you very much. 🙂 ... View more

Take a look at $SPLUNK_HOME/etc/system/default/segmentors.conf. You can add your own key/value segmentors by creating that file in ./local/. As always, test, test, test before deploying into production:) You can also check out http://docs.splunk.com/Documentation/Splunk/latest/admin/Segmentersconf ... View more

In props.conf insert: SEDCMD=s/(?sm)Key Length: 0(.+)//g That should remove everythign after "Key Length: 0" ... View more

Hi lguinn, can you provide us steps on how to reduce the size of the index. thanks in advance. ... View more

@venkatesh296 - Did one of the answers below help provide a solution your question? If yes, please click “Accept” below the best answer to resolve this post and up-vote any answers that were helpful. If no, please leave a comment with more feedback. Thanks. ... View more

You can create your own roles, and edit existing roles. Do you have something more specific you're looking to do? If you mean userA can see only one glass table, than you can create a role that has required functionality and limit that specific role using the permissions for the glass table(s) in question. Take a look at: https://docs.splunk.com/Documentation/ITSI/2.4.1/Configure/Configureusersandroles ... View more

No I don't need the DBNAME as that is not part of the Message string. I used this regex REGEX = [.*]\s+(?P[A-Z].+) This appears to be working. ... View more

did you ever figure the issue out? I have the same error and can't grant access to Box thx ... View more

The FIELDS = needs to be a quoted string list as in: FIELDS = "status","message_id","message_delivered_date","message_delivered_time","service","payload" I see that difference and Ido not have DATETIME_CONFIG = CURRENT. I would remove that setting anyway, as the docs say this about it: DATETIME_CONFIG = * Specifies which file configures the timestamp extractor, which identifies timestamps from the event text. * This configuration may also be set to "NONE" to prevent the timestamp extractor from running or "CURRENT" to assign the current system time to each event. * "CURRENT" will set the time of the event to the time that the event was merged from lines, or worded differently, the time it passed through the aggregator processor. ... View more

Try this one: REGEX= (?ism)"(\w+)":"([^"]+)|timestamp":(?P\d+) FORMAT = $1:$2 ... View more

We have already configured the Ironport feeds with a rename of the sourcetype to cisco:esa:legacy and linked it to the CIM model so any change is not possible. I have approached Splunk Support and they have referred me back to Splunk Answers since this is not a break fix. However, this is not working as prescribed so we are looking for some help to resolve this issue ... View more