Splunk Enterprise

How do I Insert custom field and value to splunk?

jordilazo
Explorer

Hi,

Im dummy in Splunk and I have one doubt. Maybe you can help me.

I want to insert in an index that I have created some data that I have obtained when executing a script in python, so the result of the script is the following:

 

 

sourcetype="script_emails" mail_sender="jordi@jordilazo.com" mail_recipient="jordilazo2@jordilazo.es" mail_date="10-10-2022" mail_subject="RE: NMXWZFOG< >VSTI" mail_reviewcomment="Comment:ÑC<AZR=@P"&"\A"

 

 

How do I configure the inputs, props and transform so that it is uploaded correctly in Splunk?

- Field - Value
- Source
- Sourcetype

I have this:

 

inputs.conf

 

 

[script://"script.py"]
disabled = 0
index = python_emails
interval = 22 13 * * *
source = ????(I dont know what to insert here)
sourcetype = mytest

 

 

 

transform.conf

 

 

[test_sourcetype]
REGEX = sourcetype="(\w+)"
FORMAT = sourcetype::$1
DEST_KEY = MetaData:Sourcetype

[test_comment]
REGEX = mail_reviewcomment="(.+)"
FORMAT = mail_reviewcomment::$1
WRITE_META = true

 

 

 

props.conf

 

 

[mytest]
SHOULD_LINEMERGE = false
NO_BINARY_CHECK = true
TIME_PREFIX = timestamp=
MAX_TIMESTAMP_LOOKAHEAD = 10
CHARSET = UTF-8
KV_MODE = auto
TRANSFORMS-test_sourcetype = test_sourcetype,test_comment

 

 

Thanks for you help!

Labels (2)
0 Karma
1 Solution

isoutamo
SplunkTrust
SplunkTrust

Hi

If you want you can add source and sourcetype to inputs.conf. If not then splunk use script name for those.

f you have several source types returned by this script (which actually means that output of this script is different) then you should use those transforms.conf settings. You can also look INGEST_EVAL command to pick up those values from input stream.

If/when you are not setting sourcetype on inputs.conf you must change in [mytest] to [source::.../<script.py>] or [script.py]  to match those events.  

You have defined that timestamp (_time) has picked from field timestamp, but I don't see it on your script output. If it's missing then ok otherwise change the filename for it (mail_date?) and check that you have correctly formatted TIME_FORMAT for it.

Test the script by running it on source host as "sudo -u<splunk user> /opt/splunk/bin/splunk cmd /path/to/your/script". This must work to use it as an scripted inputs.  Maybe you need to change the path in your inputs.conf stanza if it cannot find it by "script.py".

r. Ismo

View solution in original post

0 Karma

isoutamo
SplunkTrust
SplunkTrust

Hi

If you want you can add source and sourcetype to inputs.conf. If not then splunk use script name for those.

f you have several source types returned by this script (which actually means that output of this script is different) then you should use those transforms.conf settings. You can also look INGEST_EVAL command to pick up those values from input stream.

If/when you are not setting sourcetype on inputs.conf you must change in [mytest] to [source::.../<script.py>] or [script.py]  to match those events.  

You have defined that timestamp (_time) has picked from field timestamp, but I don't see it on your script output. If it's missing then ok otherwise change the filename for it (mail_date?) and check that you have correctly formatted TIME_FORMAT for it.

Test the script by running it on source host as "sudo -u<splunk user> /opt/splunk/bin/splunk cmd /path/to/your/script". This must work to use it as an scripted inputs.  Maybe you need to change the path in your inputs.conf stanza if it cannot find it by "script.py".

r. Ismo

0 Karma

jordilazo
Explorer

Hi isoutamo,

Thanks for your information!!

Thanks to you I have been able to upload the data correctly except for a small error.
In the field: mail_reviewcomment I have a = which makes splunk automatically create a new field for me without me asking it.
Is there any way to be able to insert the = symbol to the splunk and at the same time not create a new field? Thank you.

0 Karma

isoutamo
SplunkTrust
SplunkTrust

I think that you must add search time extraction for that field, otherwise it use = as key value separator. Just add it to search head with props.conf or with gui.

0 Karma

jordilazo
Explorer

Sorry is the first time I am doing this.

Could you explain it with more detail?

Where should I add EXTRACT? as you said in the props.conf but exactly where and with what parameters?

0 Karma
Get Updates on the Splunk Community!

Index This | I am a number, but when you add ‘G’ to me, I go away. What number am I?

March 2024 Edition Hayyy Splunk Education Enthusiasts and the Eternally Curious!  We’re back with another ...

What’s New in Splunk App for PCI Compliance 5.3.1?

The Splunk App for PCI Compliance allows customers to extend the power of their existing Splunk solution with ...

Extending Observability Content to Splunk Cloud

Register to join us !   In this Extending Observability Content to Splunk Cloud Tech Talk, you'll see how to ...