Splunk Dev

How do you run a Python script on (or before) an index?

agro1986001
Engager

Hi,

Is it possible to create a custom app on Splunk so that will run a Python script on a custom source (or sourcetype) before a new item is indexed? Specifically, I would also like to access the data that is incoming.

Suppose I have this event coming into splunk:

eventName=newUser firstName=henry lastName=adams

I would like to intercept it and then perhaps add fullName="henry adams"

PS: on my use case, I have to do the processing on/before index, so I cannot use real time alerts.

Best regards

Labels (1)
0 Karma

rameshprasad
New Member

Hi, I have a similar requirement where I want to intercept the event and want to modify the value of a field which will again come from a REST call. Basically I want to execute a script before sending the fields to index. I am getting data through HTTP Event Collector. Is this possible to do in Splunk?

0 Karma

vishaltaneja070
Motivator

Hello @agro1986001
I think the below example can be achieved using props and transform using regex

In Splunk using regex, you can replace the data inside the event.

https://docs.splunk.com/Documentation/Splunk/7.2.3/Data/Configureindex-timefieldextraction
0 Karma

agro1986001
Engager

Hi @vishaltaneja07011993

I gave a simple example of reading data, but unfortunately what I'm doing is not just that. Let's say for example that my python script wants to write to a database (mysql, redis, etc.), which cannot be done using just splunk (only an example. the point is I really want a python script to be called). I want to know whether it's technically possible or not.

Thanks a lot!

0 Karma

vishaltaneja070
Motivator

@agro1986001

Okay. Yes you can call python script through splunk using inputs.conf.

https://docs.splunk.com/Documentation/Splunk/7.2.3/AdvancedDev/ScriptedInputsIntro

And secondly, if we forward data to database from Splunk, you can relay on db connect as well.

0 Karma

agro1986001
Engager

Thanks, but that's different than what I want to accomplish.

I'm not trying to make a script that inputs data to splunk.

I already have data flowing into splunk. I just want a script to be called for every event before that event gets indexed.

0 Karma

vishaltaneja070
Motivator

@agro1986001

Sorry that doesn't seem to feasible using Splunk.

After indexing, i think still it is possible if you save it as alert but not before indexing.

0 Karma
Get Updates on the Splunk Community!

Detecting Remote Code Executions With the Splunk Threat Research Team

WATCH NOWRemote code execution (RCE) vulnerabilities pose a significant risk to organizations. If exploited, ...

Enter the Splunk Community Dashboard Challenge for Your Chance to Win!

The Splunk Community Dashboard Challenge is underway! This is your chance to showcase your skills in creating ...

.conf24 | Session Scheduler is Live!!

.conf24 is happening June 11 - 14 in Las Vegas, and we are thrilled to announce that the conference catalog ...