Getting Data In

How to make changes so that all logs should be indexed in a proper format?

alex4
Loves-to-Learn Lots

I am getting logs in Splunk. But the logs are in improper format. So I want to make changes so that all my logs should be indexed in a proper format.

Below are the format of the logs. Please help me regex in props & transforms.conf

 

 

2022-12-15T16:02:11+05:30 gd9017 msgtra.imss[26879]: NormalTransac#0112022 Dec 15 16:01:30 +05:30#0112022/12/15 16:01:31 +05:30#0112022 Dec 15 16:01:31 +05:30#01136082476.4647.1671100216806.JavaMail.jwsuser@communication-api-9-xrc8m#0118B3D3323-EFDB-5B05-A5EA-9077D10C03DD#011288C06408D#0111#011donotreply@test.com#011uat08@test.org.in#011Invoices not transmitted to ICEGATE because of Negative ledger balance.#011103.83.79.99#011[172.18.201.13]:25#011250 2.0.0 Ok: queued as 619AE341807#011sent#01100100000000000000#0110#011#0112022 Dec 15 16:01:31 +05:30#0112022 Dec 15 16:01:31 +05:30#011#0113#011

 

 

Fields in the logs are time, computer, from, to, subjectline, attachment name

 

Labels (2)
0 Karma

richgalloway
SplunkTrust
SplunkTrust

What is improper about the current onboarding format?  What are the current props.conf settings?

There are several timestamps in the sample event.  Please identify the one to use for _time.  Similarly, please point out the computer, from, to, subjectline, and attachment name fields.

---
If this reply helps you, Karma would be appreciated.
0 Karma

m_pham
Splunk Employee
Splunk Employee

Just adding to richgalloway's comment:

- What is generating that log? You can most likely look up documentation of the field names of all the values in the log. Or you can ask the person managing the tech generating the log for more information.

- You will have to create a props and transforms conf files to extract the field pair values on the search head - it looks like the field values are delimited by the "#" sign. The link below can be used as a starting point: 

https://docs.splunk.com/Documentation/SplunkCloud/latest/Knowledge/ExtractfieldsinteractivelywithIFX...

 

0 Karma
Get Updates on the Splunk Community!

Enterprise Security Content Update (ESCU) | New Releases

In December, the Splunk Threat Research Team had 1 release of new security content via the Enterprise Security ...

Why am I not seeing the finding in Splunk Enterprise Security Analyst Queue?

(This is the first of a series of 2 blogs). Splunk Enterprise Security is a fantastic tool that offers robust ...

Index This | What are the 12 Days of Splunk-mas?

December 2024 Edition Hayyy Splunk Education Enthusiasts and the Eternally Curious!  We’re back with another ...