Getting Data In

How do I use props.conf and or transforms.conf to parse log file?

eholz1
Contributor

I would like to use props.conf and/or transforms.conf to parse data coming from a generic

single line log file using regex to search for "Error" or "Notice" I did a test for my regex in regex 101, and the regex seems ok. regex = (?<=Error)(.*$)

I do have a sourcetype for the incoming data - what should I be looking for and what files should I edit to

allow this?

Thanks,

eholz1

Labels (2)
0 Karma
1 Solution

richgalloway
SplunkTrust
SplunkTrust

There is nothing to stop you from doing that, but it may not do what you expect.

If you need to extract both Error and Notice fields then use two EXTRACT settings.

EXTRACT-error = Error(?<Error>.*)$
EXTRACT-notice = Notice(?<Notice>.*)$
---
If this reply helps you, Karma would be appreciated.

View solution in original post

richgalloway
SplunkTrust
SplunkTrust

To parse that one field at search-time (the best option, IMO), you can use an EXTRACT setting in props.conf.  No transform is needed.

[mysourcetype]
EXTRACT-error = Error(?<Error>.*)$

Note the required named capture group in the regex.  Also, avoid using lookbehind in regexes as they're expensive and usually unnecessary.

---
If this reply helps you, Karma would be appreciated.

eholz1
Contributor

hello richgallowy,

I will give this a shot, and see what happens.

and come back if I have any questions.  could I add an "OR" to this like: 

EXTRACT-error = Error(?<Error>.*)$

OR Notice(?<Notice>.*)$

thanks for the support,

eholz1

0 Karma

richgalloway
SplunkTrust
SplunkTrust

There is nothing to stop you from doing that, but it may not do what you expect.

If you need to extract both Error and Notice fields then use two EXTRACT settings.

EXTRACT-error = Error(?<Error>.*)$
EXTRACT-notice = Notice(?<Notice>.*)$
---
If this reply helps you, Karma would be appreciated.

eholz1
Contributor

Thanks again for the tips.

Now one last dump question - how do I verify that this works?

Do I need any other settings in the props.conf file to allow the data to be viewed in a search?

When I configure the props.conf file with the source "my_source", I still see all the data in the log file that i do not need.

 

Thanks again,

eholz1

0 Karma

richgalloway
SplunkTrust
SplunkTrust

You'll know the extractions work when you see the new fields on the left side of the results or when you see values from | table Error Notice.

Yes, you will still see field "you do not need".  Nothing in the OP asked about getting rid of anything.  You probably don't want to get rid of anything at this points, because you may need it for another use case.  You can, however, only show what you need by using the table command as above.

---
If this reply helps you, Karma would be appreciated.

eholz1
Contributor

Thanks again for the clarification - I am still learning!

0 Karma
Get Updates on the Splunk Community!

Splunk Observability Cloud’s AI Assistant in Action Series: Analyzing and ...

This is the second post in our Splunk Observability Cloud’s AI Assistant in Action series, in which we look at ...

Elevate Your Organization with Splunk’s Next Platform Evolution

 Thursday, July 10, 2025  |  11AM PDT / 2PM EDT Whether you're managing complex deployments or looking to ...

Splunk Answers Content Calendar, June Edition

Get ready for this week’s post dedicated to Splunk Dashboards! We're celebrating the power of community by ...