Getting Data In

How to make changes so that all logs should be indexed in a proper format?

alex4
Loves-to-Learn Lots

I am getting logs in Splunk. But the logs are in improper format. So I want to make changes so that all my logs should be indexed in a proper format.

Below are the format of the logs. Please help me regex in props & transforms.conf

 

 

2022-12-15T16:02:11+05:30 gd9017 msgtra.imss[26879]: NormalTransac#0112022 Dec 15 16:01:30 +05:30#0112022/12/15 16:01:31 +05:30#0112022 Dec 15 16:01:31 +05:30#01136082476.4647.1671100216806.JavaMail.jwsuser@communication-api-9-xrc8m#0118B3D3323-EFDB-5B05-A5EA-9077D10C03DD#011288C06408D#0111#011donotreply@test.com#011uat08@test.org.in#011Invoices not transmitted to ICEGATE because of Negative ledger balance.#011103.83.79.99#011[172.18.201.13]:25#011250 2.0.0 Ok: queued as 619AE341807#011sent#01100100000000000000#0110#011#0112022 Dec 15 16:01:31 +05:30#0112022 Dec 15 16:01:31 +05:30#011#0113#011

 

 

Fields in the logs are time, computer, from, to, subjectline, attachment name

 

Labels (2)
0 Karma

richgalloway
SplunkTrust
SplunkTrust

What is improper about the current onboarding format?  What are the current props.conf settings?

There are several timestamps in the sample event.  Please identify the one to use for _time.  Similarly, please point out the computer, from, to, subjectline, and attachment name fields.

---
If this reply helps you, Karma would be appreciated.
0 Karma

m_pham
Splunk Employee
Splunk Employee

Just adding to richgalloway's comment:

- What is generating that log? You can most likely look up documentation of the field names of all the values in the log. Or you can ask the person managing the tech generating the log for more information.

- You will have to create a props and transforms conf files to extract the field pair values on the search head - it looks like the field values are delimited by the "#" sign. The link below can be used as a starting point: 

https://docs.splunk.com/Documentation/SplunkCloud/latest/Knowledge/ExtractfieldsinteractivelywithIFX...

 

0 Karma
Get Updates on the Splunk Community!

Announcing Scheduled Export GA for Dashboard Studio

We're excited to announce the general availability of Scheduled Export for Dashboard Studio. Starting in ...

Extending Observability Content to Splunk Cloud

Watch Now!   In this Extending Observability Content to Splunk Cloud Tech Talk, you'll see how to leverage ...

More Control Over Your Monitoring Costs with Archived Metrics GA in US-AWS!

What if there was a way you could keep all the metrics data you need while saving on storage costs?This is now ...