Getting Data In

discard few fields and ingest required data using scripted input

ips_mandar
Builder

I want to discard few fields from monitoring input so not increase license usage
What will be best way to do it
It can be possible with SEDCMD but I am trying to know using scripted input
I am newbie in script writing ..can anyone guide me to write python script to take only required data in splunk.
What are the stepsto follow?
Thanks in advance. I am using Splunk 7.3 on Windows server.

0 Karma

woodcock
Esteemed Legend

If you are using a scripted input, then you can either edit the script to modify what it outputs, or, if you already have a SEDCMD that works, you can just add | sed "Your SEDCMD here" to the end of the command line.

ips_mandar
Builder

Thanks @woodcock
For example I have below props.conf

SEDCMD-aremoveheader = s/^\<\?xml[^\>]*\>\n*//g

Then What I need to write in script to run above in script(will it by .py?).
Note: the above props.conf is in Indexer and if I run only |sed ""using script it will not fetch the data from remote server. Since I want to fetch data from remote server.

0 Karma

woodcock
Esteemed Legend

You have something like this in your inputs.conf:

 [script:///path/to/your_script.sh]

Change it to this:

[script:///path/to/your_script.sh | sed "s/^\<\?xml[^\>]*\>\n*//g"]
0 Karma

woodcock
Esteemed Legend

You might need to specify the full path to the sed binary.

0 Karma

richgalloway
SplunkTrust
SplunkTrust

Splunk will index whatever a scripted input writes to stdout. Your script can read any data at all, but the key is write only the fields you want in Splunk.

---
If this reply helps you, Karma would be appreciated.

ips_mandar
Builder

Thanks @richgalloway Can you please help me with sample script like python . for example I have csv file in which I want only field 2 ,field 3 ,field 5 to be extracted... Since I never written any script can you please help to provide sample script which will work like mentioned above.

0 Karma

richgalloway
SplunkTrust
SplunkTrust

Google can provide lots of examples. Here's one I crafted from the first result.

import csv

with open('my_csv.txt', mode='r') as csv_file:
    csv_reader = csv.DictReader(csv_file)
    for row in csv_reader:
        print(f'{row[2]},{row[3]},{row[5]}')
---
If this reply helps you, Karma would be appreciated.
0 Karma

ips_mandar
Builder

Thanks @richgalloway I will give this try and will keep posted.

0 Karma
Got questions? Get answers!

Join the Splunk Community Slack to learn, troubleshoot, and make connections with fellow Splunk practitioners in real time!

Meet up IRL or virtually!

Join Splunk User Groups to connect and learn in-person by region or remotely by topic or industry.

Get Updates on the Splunk Community!

Mastering Threat Intelligence in ES 8.5, Splunk AI Assistant v2, and More from Splunk ...

Splunk Lantern is Splunk’s customer success center that provides practical guidance from Splunk experts on key ...

Break the Build: Inside the KubeDoom Lounge at .conf26

    You step up to the machine. The pixelated corridors of a certain 1993 FPS load in front of you, EMP Pulse ...

Splunk Auto Ingestion Parallel Pipeline Scaling

Why this feature matters Many Splunk environments experience ingestion pressure long before the host is fully ...