Getting Data In

How to make changes so that all logs should be indexed in a proper format?

alex4
Loves-to-Learn Lots

I am getting logs in Splunk. But the logs are in improper format. So I want to make changes so that all my logs should be indexed in a proper format.

Below are the format of the logs. Please help me regex in props & transforms.conf

 

 

2022-12-15T16:02:11+05:30 gd9017 msgtra.imss[26879]: NormalTransac#0112022 Dec 15 16:01:30 +05:30#0112022/12/15 16:01:31 +05:30#0112022 Dec 15 16:01:31 +05:30#01136082476.4647.1671100216806.JavaMail.jwsuser@communication-api-9-xrc8m#0118B3D3323-EFDB-5B05-A5EA-9077D10C03DD#011288C06408D#0111#011donotreply@test.com#011uat08@test.org.in#011Invoices not transmitted to ICEGATE because of Negative ledger balance.#011103.83.79.99#011[172.18.201.13]:25#011250 2.0.0 Ok: queued as 619AE341807#011sent#01100100000000000000#0110#011#0112022 Dec 15 16:01:31 +05:30#0112022 Dec 15 16:01:31 +05:30#011#0113#011

 

 

Fields in the logs are time, computer, from, to, subjectline, attachment name

 

Labels (2)
0 Karma

richgalloway
SplunkTrust
SplunkTrust

What is improper about the current onboarding format?  What are the current props.conf settings?

There are several timestamps in the sample event.  Please identify the one to use for _time.  Similarly, please point out the computer, from, to, subjectline, and attachment name fields.

---
If this reply helps you, Karma would be appreciated.
0 Karma

m_pham
Splunk Employee
Splunk Employee

Just adding to richgalloway's comment:

- What is generating that log? You can most likely look up documentation of the field names of all the values in the log. Or you can ask the person managing the tech generating the log for more information.

- You will have to create a props and transforms conf files to extract the field pair values on the search head - it looks like the field values are delimited by the "#" sign. The link below can be used as a starting point: 

https://docs.splunk.com/Documentation/SplunkCloud/latest/Knowledge/ExtractfieldsinteractivelywithIFX...

 

0 Karma
Get Updates on the Splunk Community!

Detecting Remote Code Executions With the Splunk Threat Research Team

WATCH NOWRemote code execution (RCE) vulnerabilities pose a significant risk to organizations. If exploited, ...

Enter the Splunk Community Dashboard Challenge for Your Chance to Win!

The Splunk Community Dashboard Challenge is underway! This is your chance to showcase your skills in creating ...

.conf24 | Session Scheduler is Live!!

.conf24 is happening June 11 - 14 in Las Vegas, and we are thrilled to announce that the conference catalog ...