Splunk Search

Splunk extraction of fields for email texts

tan_junyuan
Engager

I have the infra as shown below:
Splunk Log Forwarder-> Splunk Indexer

The Log forwarder defines which data goes into which index, which is configured at Log Forwarder, Data Input -> Files/Directories.
The data(email text file) goes into the directory , then the forwarder forwards to the correct index.
Note: Email texts do not conform to any Common Information Model, when it goes into splunk indexer, it will be just text.

At the Log forwarder, there is a list of props.conf and transforms.conf to tell it how to look for regexp and extract the fields .
However, if new content and regexp changes, it will not be able to detect it.

Is there any intelligent way to know what fields to extract the fields(if the lines have this format xxxx followed by semi colon)
eg. Sent: Feb 19 2019 10:00 -> reported_time(field): Feb 19 2019 10:00
Reported issue: issue1-> reported_issue(field): issue1
user:xxx -> affected_user(field): user

Is this the correct way to extract the fields at the log forwarder?
Or it could be done at the Indexer, so that if the regexp changes, I just need to update, and the fields will be re-extracted accordingly.

Tags (1)
0 Karma

vinod94
Contributor

Hi @tan_junyuan ,

You need to do it at the indexer level, i mean in the indexer.You should not make any changes in forwarder (not a good practice).

This doc might help you!

https://docs.splunk.com/Documentation/Splunk/7.2.4/Knowledge/Exampleconfigurationswithprops.conf

0 Karma
Get Updates on the Splunk Community!

See just what you’ve been missing | Observability tracks at Splunk University

Looking to sharpen your observability skills so you can better understand how to collect and analyze data from ...

Weezer at .conf25? Say it ain’t so!

Hello Splunkers, The countdown to .conf25 is on-and we've just turned up the volume! We're thrilled to ...

How SC4S Makes Suricata Logs Ingestion Simple

Network security monitoring has become increasingly critical for organizations of all sizes. Splunk has ...