Splunk Dev

How do you run a Python script on (or before) an index?

agro1986001
Engager

Hi,

Is it possible to create a custom app on Splunk so that will run a Python script on a custom source (or sourcetype) before a new item is indexed? Specifically, I would also like to access the data that is incoming.

Suppose I have this event coming into splunk:

eventName=newUser firstName=henry lastName=adams

I would like to intercept it and then perhaps add fullName="henry adams"

PS: on my use case, I have to do the processing on/before index, so I cannot use real time alerts.

Best regards

Labels (1)
0 Karma

rameshprasad
New Member

Hi, I have a similar requirement where I want to intercept the event and want to modify the value of a field which will again come from a REST call. Basically I want to execute a script before sending the fields to index. I am getting data through HTTP Event Collector. Is this possible to do in Splunk?

0 Karma

vishaltaneja070
Motivator

Hello @agro1986001
I think the below example can be achieved using props and transform using regex

In Splunk using regex, you can replace the data inside the event.

https://docs.splunk.com/Documentation/Splunk/7.2.3/Data/Configureindex-timefieldextraction
0 Karma

agro1986001
Engager

Hi @vishaltaneja07011993

I gave a simple example of reading data, but unfortunately what I'm doing is not just that. Let's say for example that my python script wants to write to a database (mysql, redis, etc.), which cannot be done using just splunk (only an example. the point is I really want a python script to be called). I want to know whether it's technically possible or not.

Thanks a lot!

0 Karma

vishaltaneja070
Motivator

@agro1986001

Okay. Yes you can call python script through splunk using inputs.conf.

https://docs.splunk.com/Documentation/Splunk/7.2.3/AdvancedDev/ScriptedInputsIntro

And secondly, if we forward data to database from Splunk, you can relay on db connect as well.

0 Karma

agro1986001
Engager

Thanks, but that's different than what I want to accomplish.

I'm not trying to make a script that inputs data to splunk.

I already have data flowing into splunk. I just want a script to be called for every event before that event gets indexed.

0 Karma

vishaltaneja070
Motivator

@agro1986001

Sorry that doesn't seem to feasible using Splunk.

After indexing, i think still it is possible if you save it as alert but not before indexing.

0 Karma
Get Updates on the Splunk Community!

Introducing the 2024 SplunkTrust!

Hello, Splunk Community! We are beyond thrilled to announce our newest group of SplunkTrust members!  The ...

Introducing the 2024 Splunk MVPs!

We are excited to announce the 2024 cohort of the Splunk MVP program. Splunk MVPs are passionate members of ...

Splunk Custom Visualizations App End of Life

The Splunk Custom Visualizations apps End of Life for SimpleXML will reach end of support on Dec 21, 2024, ...