topic Re: Filter & Ingest Data in Getting Data In

Filter & Ingest Data

Karthikeyan — Tue, 17 Aug 2021 15:22:20 GMT

Hi Experts,

I have specific requirement to split the contents of a file and ingest it as a separate events. In that events, a filter to be applied and ingest the filtered data to Splunk indexer.

I have created a REGEX pattern which split the contents of the file and ingesting the data in to separate events as desired. Now, my issue is with the filtering of ingested data. In each event, I need to filter fro AGGREGATED_EXECUTION and ingest only the event which has that content. I set the configuration as below.

props.conf:
[expensive_statements]
TRANSFORMS-set= send_events

transforms.conf:
[send_events]
REGEX = AGGREGATED_EXECUTION
DEST_KEY = queue
FORMAT = indexQueue

Above settings is made on HF. Still the filtering is not happening as expected. Kindly help in resolving the issue with filtering.

Regards, Karthikeyan.SV

Re: Filter & Ingest Data

codebuilder — Wed, 18 Aug 2021 18:52:47 GMT

Did you either restart the SHC/SH after making the changes?

Or you can run the following in the search bar to get the same results:

|rest /services/authentication/users splunk_server=local

Re: Filter & Ingest Data

Karthikeyan — Thu, 19 Aug 2021 12:59:32 GMT

I restarted SH after making the changes. Still the new props is not effected.

Re: Filter & Ingest Data

shivanshu1593 — Thu, 19 Aug 2021 16:13:33 GMT

Instead of re-starting splunkd on the search head, try restarting the splunkd service on the Heavy Forwarder, where the changes have been made and then check if the results are effective.

If not, btool can tell if the configuration is really being loaded in the memory or not.

Thanks,

****If it helped, please upvote and accept it as a solution. It helps others to find the solution more quickly in the future****