You have two options: DBConnect - You can bring audit logs  from Oracle DBA_AUDIT_TRAIL table.  Make sure the DBA has configured to send the audit logs to DBA_AUDIT_TRAIL.  If other RDS databases have option to store audit logs to a table/view you can also bring the logs via dbconnect. AWS RDS Logs -- Have AWS Cloudwatch collect the RDS logs put them in an S3 bucket.  Bring the RDS logs to Splunk via SQS-Based S3 inputs using Splunk Add-on for AWS.  You can then build a parser for search - time and display the result in a dashboard. ... View more

Sorry for such a late reply.  You can use either DBConnect or send the rds logs via AWS Cloudwatch and bring the data to Splunk using AWS TA for Splunk using sqs based s3 inputs. ... View more

Specify the sourcetype at the oneshot command and have a props.conf with the following paramers set.  The TIME parameters will take care of your timestamp issue.  Make sure to restart the splunkd service after adding the props.conf. [sourcetypename] LINE_BREAKER TIME_PREFIX MAX_TIMESTAMP_LOOKAHEAD TIME_FORMAT TRUNCATE SHOULD_LINEMERGE = false # LINE_BREAKER should be properly set so you can keep SHOULD_LINEMERGE = false NO_BINARY_CHECK = true ... View more

We need to remember that Universal Forward does not do index-time parsing, on the other hand a Splunk Enterprise server such as a Heavy Forwarder, Search Head, Deployment server, depoyer, etc does the index-time parsing.  Indexers  expect that the data that come via another Splunk Enterprise server have the data cooked. ... View more

Correct.  We have to make sure that changing the configs via cli editor we need to restart the splunkd service for the configs to take effect. ... View more

We need to keep in mind that KV_MODE applies to search time only and the field extractons ae best to be done at search time.  Therefore, at index time if you have the following parameters set you should be good. props.conf [sourcetypename] LINE_BREAKER TIME_PREFIX MAX_TIMESTAMP_LOOKAHEAD TIME_FORMAT TRUNCATE SHOULD_LINEMERGE = false # LINE_BREAKER should be properly set so you can keep SHOULD_LINEMERGE = false NO_BINARY_CHECK = true  ... View more

Correct. This is also known as the De Morgan's Law of Union Or De Morgan's Law of Intersection ... View more

This part is very important "any that did not match in the last lookup would null out previous matches." ... View more

sourcetype in props.conf in case-senstive [MySourcetype] is different from [mysourcetype] From Splunk Documentation (pros.conf) By default, [source::<source>] and [<sourcetype>] stanzas match in a case-sensitive manner, while [host::<host>] stanzas match in a case-insensitive manner. This is a convenient default, given that DNS names are case-insensitive. ... View more

Thanks so much for your post.  I am surprised that the following did not work.  "_auto" is not the default value meaning setting it  "_auto" would make Splunk automatically extract the archived file unless a file extension is required.  I am having a case where Splunk is ingesting gzip file without extension but the files after ingestion is not in text format.  After testing a file with .gz Splunk recognized it and decompressed it properly.  That tells me that Splunk requires an archived file to have an extension. unarchive_cmd = _auto   ... View more

You can use deployer_push_mode = full | merge_to_default | local_only | default_only You have add it under [shclustering] stanza in app.conf of the app you are deploying.  I think you have to have Splunk 7.3.0 and higher. Here's the doc. https://docs.splunk.com/Documentation/Splunk/8.0.1/DistSearch/PropagateSHCconfigurationchanges ... View more

If you provide all the capabilities to a user level role it still won't work.  I gave all the capabilites--I mean all still the user didn't have | rest splunk_server=local /services/authentication/ capabilities.  Only the Admin role has. ... View more

One thing to keep in mind is that for a Splunk Forwarder .../etc/system/local takes precedence over /etc/apps directory.  So a deploymentclient.conf residing in /etc/system/local will take precedence over deploymentclient.conf in /etc/apps directory.  It is not a best practice to make a deploymentclient.conf app available from Deployment Server for the forwarders to download since it will be downloaded to /etc/apps directory. ... View more

Nice examples, Woodcock!!!  Eventtype is quite easy to understand but tag with enabled/disabled <field><value> is not always clear to a lot of people.    On, the other hand, the whole eventtype can also be tagged in tags.conf like the following [eventtype=LAB_EVENTS] lab = enabled ... View more

It should read as woodcock mentioned. "Although the execution does not "stop early" fields are not overridden so once an earlier-executed transform has given a field a value, later-executed ones CAN update/overwrite that original value."   ... View more

Since this will be performed at index parsing stage file precedence will be in global context so /system/local will have higher precedence than application/local.  So, in your case "my_sourcetype" will be created first then you can use "my_sourcetype" in application/local to redirect logs to different indexes.   ... View more

Good point the EXTRACT, REPORT and TRANSFORMS class names must be unique  (i.e TRANSFORMS-abc and TRANSFORMS-xyz and NOT  TRANSFORMS-abc and  TRANSFORMS-abc)     ... View more

This method works in 7.1+.  It worked for me. ... View more

Darrenfuller's answer is good. There are advantages and disadvantages in index time and search time field extraction for csv file with header. Search Time : Less storage space for indexed data than index time extraction Index Time: If a field's position changes, and it can happen sometimes, then creating a new report class for search time will override some of the old fields. Here is a link to a case study on search time vs index time for json file. https://www.hurricanelabs.com/blog/splunk-case-study-indexed-extractions-vs-search-time-extractions ... View more

As of today (April 29, 2020) post process panels export button is still greyed out. ... View more

I give klaxdal a plus point because it works ... View more

Is that a forwarder on the syslog server? ... View more

I have not tried it but isn't "remote_timeline_fetchall = false" good for reporting searches instead of search that you run on search window? ... View more

In a cluster environment, you should already have a copy of the searchable bucket in another indexer provided you have at least SF=2. Enable the indexer cluster maintenance mode Stop the indexer in question 3. a. Move the broken journal file away (while splunkd turned off) to another place or b. Delete the bucket Start the indexer in question Disable the indexer cluster maintenance mode. ... View more

Good information. If you are in a cluster environment. 1. Enable the indexer cluster maintenance mode 2. Stop the indexer in question 3. Follow the above steps 1 through 5 4. Start the indexer in question 5. Disable the indexer cluster maintenance mode. ... View more