topic Re: Ingest Action to include message in Getting Data In

Ingest Action to include message

yuvaraj_m91 — Thu, 09 May 2024 05:56:24 GMT

I am using ingest action to filter the log message before being indexed in splunk..

I want to include the message that matches only the keyword :ERROR: and :FATAL: rest all of the messages should not be indexed.

Whereas in splunk ingest action has the filter to only exclude message not the include

Re: Ingest Action to include message

deepakc — Thu, 09 May 2024 07:24:37 GMT

You might then be able to apply a regex pattern to say to NOT not match ERROR or FATAL, therefore keep them, and discard the rest.

Try this
^(?!.*(ERROR|FATAL)).*$

Re: Ingest Action to include message

yuvaraj_m91 — Thu, 09 May 2024 07:56:37 GMT

I tried this but still i am seeing other events being ingested apart from :ERROR: and :FATAL:

Re: Ingest Action to include message

PickleRick — Thu, 09 May 2024 08:00:17 GMT

If you're not hellbent of doing it with Ingest Actions, you can just use transforms to filter out all events except for the ones you want

https://docs.splunk.com/Documentation/Splunk/latest/Forwarding/Routeandfilterdatad

In your case you'd need to first have a "match-all" transform rerouting all data to nullQueue, and then a transform maching only ERROR/FATAL events sending the events to indexQueue.

Re: Ingest Action to include message

yuvaraj_m91 — Thu, 09 May 2024 08:04:16 GMT

we are using splunk cloud UI

Re: Ingest Action to include message

PickleRick — Thu, 09 May 2024 08:05:36 GMT

Doesn't matter. You can make an app with those settings and deploy it to your Cloud instance.

Re: Ingest Action to include message

deepakc — Thu, 09 May 2024 08:41:04 GMT

Suggestions made by @PickleRick are probably best to go with.

In terms of it still not working - you will most likely need to adjust the reg-ex pattern based on your logs.