Getting Data In

How to make changes so that all logs should be indexed in a proper format?

alex4
Loves-to-Learn Lots

I am getting logs in Splunk. But the logs are in improper format. So I want to make changes so that all my logs should be indexed in a proper format.

Below are the format of the logs. Please help me regex in props & transforms.conf

 

 

2022-12-15T16:02:11+05:30 gd9017 msgtra.imss[26879]: NormalTransac#0112022 Dec 15 16:01:30 +05:30#0112022/12/15 16:01:31 +05:30#0112022 Dec 15 16:01:31 +05:30#01136082476.4647.1671100216806.JavaMail.jwsuser@communication-api-9-xrc8m#0118B3D3323-EFDB-5B05-A5EA-9077D10C03DD#011288C06408D#0111#011donotreply@test.com#011uat08@test.org.in#011Invoices not transmitted to ICEGATE because of Negative ledger balance.#011103.83.79.99#011[172.18.201.13]:25#011250 2.0.0 Ok: queued as 619AE341807#011sent#01100100000000000000#0110#011#0112022 Dec 15 16:01:31 +05:30#0112022 Dec 15 16:01:31 +05:30#011#0113#011

 

 

Fields in the logs are time, computer, from, to, subjectline, attachment name

 

Labels (2)
0 Karma

richgalloway
SplunkTrust
SplunkTrust

What is improper about the current onboarding format?  What are the current props.conf settings?

There are several timestamps in the sample event.  Please identify the one to use for _time.  Similarly, please point out the computer, from, to, subjectline, and attachment name fields.

---
If this reply helps you, Karma would be appreciated.
0 Karma

m_pham
Splunk Employee
Splunk Employee

Just adding to richgalloway's comment:

- What is generating that log? You can most likely look up documentation of the field names of all the values in the log. Or you can ask the person managing the tech generating the log for more information.

- You will have to create a props and transforms conf files to extract the field pair values on the search head - it looks like the field values are delimited by the "#" sign. The link below can be used as a starting point: 

https://docs.splunk.com/Documentation/SplunkCloud/latest/Knowledge/ExtractfieldsinteractivelywithIFX...

 

0 Karma
Get Updates on the Splunk Community!

Splunk is Nurturing Tomorrow’s Cybersecurity Leaders Today

Meet Carol Wright. She leads the Splunk Academic Alliance program at Splunk. The Splunk Academic Alliance ...

Part 2: A Guide to Maximizing Splunk IT Service Intelligence

Welcome to the second segment of our guide. In Part 1, we covered the essentials of getting started with ITSI ...

Part 1: A Guide to Maximizing Splunk IT Service Intelligence

As modern IT environments continue to grow in complexity and speed, the ability to efficiently manage and ...