On the heavy forwarder: Create and output group: outputs.conf: [tcpout:hostGroup] server=10.20.30.40:9999 Configure a props entry for the sourcetype in question: [sourcetype_to_split] TRANSFORMS-index = hostRedirect Create the output routing transforms.conf: [hostRedirect] SOURCE_KEY = host REGEX = (host1|host2|host3|host4) IndexDEST_KEY=_TCP_ROUTING FORMAT=hostGroup ... View more

... | rex "\<pfId\>(?<pfId>\d+?)\<" | | rex "\<isoId\>(?<isoId>\d+?)\<" | search pfId=isoId | table pfId, isoId ... View more

Except......that report will only contain the jobs that haven't expired. Its based on the artifacts in the dispatch directory I believe. To get historic data, you would need to use logs. ... View more

Then the steps above should work. ... View more

It appears your events have two timestamps: Jun 19 18:02:33 192.168.100.1 Jun 19 2018 14:02:33 I normally see this when syslog receives and adds a timestamp to the events received remotely. Assuming the 14:02 timestamp is EST, then it appears the first timestamp is MST (EST + 2) Splunk always attempts to offset the time to GMT. My guess is that Splunk is applying an offset at index time, in an attempt to make the time GMT. Then its offsetting again at search time, based on your TZ preference. It appears you need to fix your props to apply the correct TZ for this source, or extract the correct timestamp. ... View more

To clarify, you are talking about collecting the system and authentication log from a linux VM. You can do this in a couple ways. Here's how a method that should be easy: On the linux VM, configure rsyslog or syslog-ng to write the events you are interested in to a log file By default, these events might already be going to /var/log/secure (authentication) and /var/log/messages (system). Install Splunk UF on the VM, and configure inputs to monitor the logs you configured. Set sourcetype to syslog Configure UF to send events to your Splunk instance. If you also want to get information from the VMware level, you will need to install the Splunk app for VMware. That app takes some setup to integrate with your VMware architecture. ... View more

Correct a terrible paste accident in the "specific user" search syntax above: index=_audit action=search search=* user=${user_of_interest} NOT "typeahead" NOT metadata NOT " | history" NOT "AUTOSUMMARY" | table _time, search ... View more

There is really no reason to use a join, the table function will join the data on the matching field: (index="xyz" RuleModule="BizRule") OR (index="xyz" client.xyz ) | rename timeTook as "Rule Response Time"| rename timeTook as procTime | table dateTime, policyNo, "Rule Response Time", procTime This assumes there is a one-to-one match based on policy number. If you expect there to be multiple events per policy number you can use stats: (index="xyz" RuleModule="BizRule") OR (index="xyz" client.xyz ) | rename timeTook as "Rule Response Time"| rename timeTook as procTime | stats max(dateTime), avg("Rule Response Time"), avg(procTime) by policyNo ... View more

I don't believe so, the cluster will not recognize changes made by edits to the ../etc/system/local/server.conf. If you're concerned...then you could shut the entire cluster down and move the location. Just requires and outage. ... View more

what is the format of the cyberArk files? Splunk uses a method to track which files have already been indexed. Splunk makes a hash of the first row of the file, to keep track of the files. If the files all have a header row, and they are identical....the hash's will be the same and Splunk will not try to index the file. If that's the case try adding the following to your the monitor stanza: initCrcLength = 4096 ... View more

Did you verify the splunk process has permissions to the read the log files you want it to monitor? Do you see any events in the $SPLUNK_HOME\var\log\splunkd.log regarding these file monitors? ... View more

Corrected text of search: (sourcetype=A AND f1=*) OR (sourcetype=B AMD f2=*) | eval requestid=coalesce(f1,f2) | table requestid user city work team ..... ... View more

Check out the coalesce function of eval (sourcetype=A AND f1=) OR (sourcetype=B AMD f2=) | eval requestid=coalesce(f1,f2) | table requestid user city work team ..... ... View more

First, NEVER edit the files in the default folders. You would want to put any changes into /opt/splunk/etc/system/local/web.conf If you edit the files in the default folders, you will loose those changes on the next upgrade/update. Making the changes in local, means those changes will persist through upgrades/updates. This setting in particular is the timeout when the front end web is connecting to the backend splunk process. At a minimum, If there is an issue between the web front-end and the splunkd backend, the session will continue to try for 5000 seconds (1.3 hours) before timing out and informing the user (or whatever process was connecting via splunk web). ... View more

You will need to do it one member at a time. Take the member out of the cluster. shut member down Follow instructions to move kvstore Start member Add member to cluster Repeat on next cluster member. ... View more

Correction, should have used back references \1 instead of variables $1 in the SEDCMD: SEDCMD-vm125620 = s/(vm-125620)\s(\w)\s(\w)\s(\w)\s(\w)\s(\w)\s(\w)\s(\w)\s(\w)\s(\w)\s(\w)/\1 \2 \3 \4 \5 \6 \7 \8 \10/ ... View more

Assuming the field at the front is the VMware eventId...you might need to create one per eventId: SEDCMD-vm125620 = s/(vm-125620)\s(\w)\s(\w)\s(\w)\s(\w)\s(\w)\s(\w)\s(\w)\s(\w)\s(\w)\s(\w)/$1 $2 $3 $4 $5 $6 $7 $8 $10/ ... View more

I misunderstood, thought you were looking to get rid of the events, not the specific fields. If you want to get rid of specific fields, you probably want to look at SEDCMD- http://docs.splunk.com/Documentation/Splunk/7.1.0/Admin/Propsconf#Field_extraction_configuration You should be able to use Sed like syntax to remove the unwanted data Something like: props.conf [sourcetype] SEDCMD-removeunwanted1 = s/{[^:]+?:[^:]+?/{/ ... View more

In your props.conf for this sourcetype, you could try using a line_breaker to split, assuming all events start with "Premise=" [sourcetype] SHOULD_LINEMERGE = false LINE_BREAKER = ([\r\n]+)Premise= ... View more

ALL messages, or just all messages of the sourcetype common? If ALL messages, not sure your issues. if messages of the sourcetype common is the issue, the problem could be your REGEX is matching more than expected. ... View more