Getting Data In

discard few fields and ingest required data using scripted input

ips_mandar
Builder

I want to discard few fields from monitoring input so not increase license usage
What will be best way to do it
It can be possible with SEDCMD but I am trying to know using scripted input
I am newbie in script writing ..can anyone guide me to write python script to take only required data in splunk.
What are the stepsto follow?
Thanks in advance. I am using Splunk 7.3 on Windows server.

0 Karma

woodcock
Esteemed Legend

If you are using a scripted input, then you can either edit the script to modify what it outputs, or, if you already have a SEDCMD that works, you can just add | sed "Your SEDCMD here" to the end of the command line.

ips_mandar
Builder

Thanks @woodcock
For example I have below props.conf

SEDCMD-aremoveheader = s/^\<\?xml[^\>]*\>\n*//g

Then What I need to write in script to run above in script(will it by .py?).
Note: the above props.conf is in Indexer and if I run only |sed ""using script it will not fetch the data from remote server. Since I want to fetch data from remote server.

0 Karma

woodcock
Esteemed Legend

You have something like this in your inputs.conf:

 [script:///path/to/your_script.sh]

Change it to this:

[script:///path/to/your_script.sh | sed "s/^\<\?xml[^\>]*\>\n*//g"]
0 Karma

woodcock
Esteemed Legend

You might need to specify the full path to the sed binary.

0 Karma

richgalloway
SplunkTrust
SplunkTrust

Splunk will index whatever a scripted input writes to stdout. Your script can read any data at all, but the key is write only the fields you want in Splunk.

---
If this reply helps you, Karma would be appreciated.

ips_mandar
Builder

Thanks @richgalloway Can you please help me with sample script like python . for example I have csv file in which I want only field 2 ,field 3 ,field 5 to be extracted... Since I never written any script can you please help to provide sample script which will work like mentioned above.

0 Karma

richgalloway
SplunkTrust
SplunkTrust

Google can provide lots of examples. Here's one I crafted from the first result.

import csv

with open('my_csv.txt', mode='r') as csv_file:
    csv_reader = csv.DictReader(csv_file)
    for row in csv_reader:
        print(f'{row[2]},{row[3]},{row[5]}')
---
If this reply helps you, Karma would be appreciated.
0 Karma

ips_mandar
Builder

Thanks @richgalloway I will give this try and will keep posted.

0 Karma
Get Updates on the Splunk Community!

.conf24 | Registration Open!

Hello, hello! I come bearing good news: Registration for .conf24 is now open!   conf is Splunk’s rad annual ...

ICYMI - Check out the latest releases of Splunk Edge Processor

Splunk is pleased to announce the latest enhancements to Splunk Edge Processor.  HEC Receiver authorization ...

Introducing the 2024 SplunkTrust!

Hello, Splunk Community! We are beyond thrilled to announce our newest group of SplunkTrust members!  The ...