How do I Insert custom field and value to splunk?

[jordilazo]

Hi,

Im dummy in Splunk and I have one doubt. Maybe you can help me.

I want to insert in an index that I have created some data that I have obtained when executing a script in python, so the result of the script is the following:

sourcetype="script_emails" mail_sender="jordi@jordilazo.com" mail_recipient="jordilazo2@jordilazo.es" mail_date="10-10-2022" mail_subject="RE: NMXWZFOG< >VSTI" mail_reviewcomment="Comment:ÑC<AZR=@P"&"\A"

How do I configure the inputs, props and transform so that it is uploaded correctly in Splunk?

- Field - Value
- Source
- Sourcetype

I have this:

inputs.conf

[script://"script.py"]
disabled = 0
index = python_emails
interval = 22 13 * * *
source = ????(I dont know what to insert here)
sourcetype = mytest

transform.conf

[test_sourcetype]
REGEX = sourcetype="(\w+)"
FORMAT = sourcetype::$1
DEST_KEY = MetaData:Sourcetype

[test_comment]
REGEX = mail_reviewcomment="(.+)"
FORMAT = mail_reviewcomment::$1
WRITE_META = true

props.conf

[mytest]
SHOULD_LINEMERGE = false
NO_BINARY_CHECK = true
TIME_PREFIX = timestamp=
MAX_TIMESTAMP_LOOKAHEAD = 10
CHARSET = UTF-8
KV_MODE = auto
TRANSFORMS-test_sourcetype = test_sourcetype,test_comment

Thanks for you help!

[isoutamo · Accepted solution]

Hi

If you want you can add source and sourcetype to inputs.conf. If not then splunk use script name for those.

f you have several source types returned by this script (which actually means that output of this script is different) then you should use those transforms.conf settings. You can also look INGEST_EVAL command to pick up those values from input stream.

If/when you are not setting sourcetype on inputs.conf you must change in [mytest] to [source::.../<script.py>] or [script.py]  to match those events.  

You have defined that timestamp (_time) has picked from field timestamp, but I don't see it on your script output. If it's missing then ok otherwise change the filename for it (mail_date?) and check that you have correctly formatted TIME_FORMAT for it.

Test the script by running it on source host as "sudo -u<splunk user> /opt/splunk/bin/splunk cmd /path/to/your/script". This must work to use it as an scripted inputs.  Maybe you need to change the path in your inputs.conf stanza if it cannot find it by "script.py".

r. Ismo

[jordilazo]

Hi isoutamo,

Thanks for your information!!

Thanks to you I have been able to upload the data correctly except for a small error.
In the field: mail_reviewcomment I have a = which makes splunk automatically create a new field for me without me asking it.
Is there any way to be able to insert the = symbol to the splunk and at the same time not create a new field? Thank you.

[isoutamo]

I think that you must add search time extraction for that field, otherwise it use = as key value separator. Just add it to search head with props.conf or with gui.

[jordilazo]

Sorry is the first time I am doing this.

Could you explain it with more detail?

Where should I add EXTRACT? as you said in the props.conf but exactly where and with what parameters?