Sorry, that won't work. The tranformations take place after timestamp extractions. ... View more

alternatively, if you want to keep some numbers, like in TEST1 or V2 , but not those sub-parts that are ONLY numbers, like _2342_ you can just alter the sed script slightly; ...| rex field=your_fieldname mode=sed "s/_\d+/_/g" ... View more

Oops, answering a little late. Hmm.. Neither the rex nor the replace will handle numbers in the middle of the strings. ... View more

You can easily do it in inline in a search with rex ; ...| rex field=your_fieldname mode=sed "s/\d//g" If you don't specify a field name, the sed script will run aginst the whole event (the _raw field). Then you can do your stats/top/chart etc on the field. http://docs.splunk.com/Documentation/Splunk/6.0.1/SearchReference/Rex /K ... View more

If your Splunk indexer is actually listening for syslog traffic itself (not really recommended) you might be in luck. From the docs for inputs.conf for UDP/TCP inputs; connection_host = [ip|dns|none] * "ip" sets the host to the IP address of the system sending the data. * "dns" sets the host to the reverse DNS entry for IP address of the system sending the data. * "none" leaves the host as specified in inputs.conf, typically the splunk system hostname. * Defaults to "ip". Thus your inputs.conf could be written like; [udp://514] connection_host=ip sourcetype = not_syslog /K ... View more

Yes, well. These operations will not change the event that is presented on screen (which is stored in the _raw field). However, they will change the extracted field values, so what happens if you run the following; index=sonicwall TemplateID=257 OR TemplateID=262 | transaction session_id startswith=TemplateID=257 | head 2 |eval dest_mac=mvfilter(dest_mac !="00:00:00:00:00:00") | table session_id dest_mac /k ... View more

Any data that is configured (by you) to have the sourcetype syslog will pass through a transformation process, where the hostname is extracted from each event, on a per event basis. Typically this is something you would want to do, if you were collecting events from an existing syslog server. Instead of having the syslog server's hostname/ip-address as originator of these events, this process will extract and rewrite the hostname from each event. This is founded on the assumption that the hostname can be found in the expected place in the message. Typically, syslog messages start with a timestamp, and that is followed by a hostname/ip-address. As you might expect, just by declaring a stream of events to be of type syslog will not make this transformation work, unless the hostname information is actually there. Also, the sourcetype name syslog has nothing to do with actual transport protocol. It just means, "I want to call these events 'syslog', and for all messages of this type, please treat them as if they were standard syslog messages, and extract the hostname from where it should be." You might want to read up on sourcetypes in general in the docs; http://docs.splunk.com/Documentation/Splunk/latest/Data/Whysourcetypesmatter One solution to deal with your current problem is to make a search-time extraction of the relevant part of the events (i.e. the part that contains the actual hostname). This can be done inline in a search query with the rex command, or through more permanent extractions in props.conf. Please post a few sample events, and you'll probably get quick help with sorting out the regular expressions (if needed). Hope this helps, /K ... View more

What type of information is represented by the XYZ1 and ABC1? Time Zone? Or something else? I don't think TIME_FORMAT can handle the regex pattern in a nice way - in fact I believe that it tries to match the literal string, square brackets and all. ... View more

Since this piece of information seems to be located in the beginning of the event, is should be fairly easy to extract it. For the sample event given above, the following would work; ... | rex "\S+\s+\S+\s+(?<hostname>\w+);" This will extract the hostname from the event, regardless of contents of the string. Then you can do the sorting/filtering/statistics as part of the query. If you need to categorize by OS, and if this information can be deduced from the hostname, you can set this up with eval functions such as case ; .... | eval category = case(match(hostname, "win"), "Windows", match(hostname,"lin"), "Linux",match(hostname,"esx"),"ESX",1=1,"Other") then do your statistics reporting with the category field. Hope this helps, K ... View more

REGEX does not work with the in sourcefield option (like EXTRACT does). Thus, the 'in ClientDetails' part of your regex is seen as a literal string to be matched. So you probably need to rewrite the regular expression so that it will work for the whole event ( _raw ). Hope this helps, K ... View more

If you want to filter it out before it is even indexed, you could use the anonymization techniques discussed in the docs (see below) to remove the faulty MAC before it even reaches the index; http://docs.splunk.com/Documentation/Splunk/6.0.1/Data/Anonymizedatausingconfigurationfiles If you want to replace it during search, i.e. after the transaction , you can use the following (assuming the multivalued field is called MAC; ... | transaction FlowID | eval MAC=mvfilter(MAC != 00:00:00:00:00:00) | or, if they always come in the same order (in this case the good MAC always come before the bad) ... | transaction FlowID | eval MAC=mvindex(MAC,0) if the bad always come before the good, use mvindex(MAC,1) See more here; http://docs.splunk.com/Documentation/Splunk/5.0.2/SearchReference/CommonEvalFunctions /k ... View more

but you'll need to sort before the streamstats ... View more

dammit, spent too long editing 🙂 ... View more

What do you mean with 'unsupported'? Unsupported by your organization or unsupported by Splunk? If the former, then just install the correct package (deb, RPM, pkg, MSI etc) or unpack the .tgz in /opt/splunkforwarder. Then configure your installtion accordingly. If the latter, you're on your own. ... View more

oops. the line wrapped on the error message, but I think you get the point. /K ... View more

Ok, this can probably be simplified a bit. The search query and the sample events do not really line up, though, so my query below may not be 100% working, but it should give you an idea on how the search can be made more readable, and hopefully more efficient. For this I will assume that no fields have been extracted, so all is done through rex (in your search it seem that you have the field object extracted, but you still extract the same info as st ). Also, there are not many common fields between the two types of event, e.g. for the failed transmissions there is no file name or size. Given the sample data, it might be possible to extract some error code or user name from the MAJOR messages instead (exemplified below). index=fxr source=*.dbg-*trc*.log MAJOR OR SUCCESS | rex "\]\s+(?<status>SUCCESS|MAJOR)\s" | rex "\s\[(?<filename>\S+)\].\sFile length (was\s)?\[(?<file_size>\d+)\]" | rex "\susername\s\[(?<user_details>[^\]]+)\]" | rex "\serror\s\[(?<error_message>[^\]]+)\]" | eval the_object = coalesce(filename, user_details) | eval the_details = coalesce(file_size, error_message) table _time status the_object the_details The result would be something like this; _time status the_object the_details ------------------------------------------------------------------------ 2013-11-11 12:23:34 SUCCESS BYGSFB20.F00 123456 2013-11-12 13:34:56 MAJOR tony@10.8.12.211:10021 org.apache.commons.net.ftp.FTPConnectionClosedException: FTP response 421 received. Server closed connection. ... View more

that takes longer time, and it reloads a bunch of other stuff. I think it's generally better suited for reloading props/transforms stuff that would normally require a restart. ... View more

While it's a very good thing to post sample events, a description of the actual problem would also be helpful. What is your current search (i.e. what is the use case)? What output do you want? /K ... View more

Did you read the following topics in the docs? http://docs.splunk.com/Documentation/Splunk/latest/Data/ConsiderationsfordecidinghowtomonitorWindowsdata http://docs.splunk.com/Documentation/Splunk/latest/Data/MonitorWindowsdata /K ... View more

Ok then search for files containing the string that matches your script name. I'm sure there is some built-in GUI tool for searching through the files on disk in windows. ... View more

Or do you have a real-time search that is still running that calls the script as an alert action? ... View more