Splunk Search

Split log entries from fixed number of characters

nowornever2
New Member

Hi,

I want to find results after timestamp in below results i.e. to separate "[2012-09-28 08:46:22,410]" & the message in each entry.

[2012-09-28 08:46:22,410] [ERROR] Device 6642(matt) encountered error. Error: ClientError: Bad Request
[2012-09-28 08:46:38,277] [WARNING] Client PPC-6DFF06C758C(7162, 7013, reported: Backup failed. Error: Quota limit reached. (#10000000e)

Please help...

Thanks and Regards,
Amit

Tags (1)
0 Karma
1 Solution

Ayn
Legend

So, to be able to filter your search results based on the log message excluding the initial timestamp, first of all extract that text to a field. This can be done with the rex command:

... | rex "^.+?\]\s+(?<logmessage>.+)"

Then simply run dedup on this field, which will cause Splunk to drop all events that have duplicate values of an event it has already seen:

... | rex "^.+?\]\s+(?<logmessage>.+)" | dedup logmessage

View solution in original post

0 Karma

Ayn
Legend

So, to be able to filter your search results based on the log message excluding the initial timestamp, first of all extract that text to a field. This can be done with the rex command:

... | rex "^.+?\]\s+(?<logmessage>.+)"

Then simply run dedup on this field, which will cause Splunk to drop all events that have duplicate values of an event it has already seen:

... | rex "^.+?\]\s+(?<logmessage>.+)" | dedup logmessage
0 Karma

Ayn
Legend

My bad, I seem to have dropped a ? sign at the beginning of the matching group. It should (?<logmessage>...), not (<logmessage>...). Editing my answer to correct that.

0 Karma

nowornever2
New Member

Thanks Ayn..
but it is giving below error::
Error in 'rex' command: The regex '^.+?]\s+(.+)' does not extract anything. It should specify at least one named group. Format: (?...).

I want to apply this criteria in search field itself when i m searching for specific logs and then remove timestamp there.Then i can download a csv and even select unique fields there also or if possible in Splunk,plz tell..

Regards,
Amit

0 Karma

nowornever2
New Member

Can you give the syntax applicable in this case...all these entries are coming in single field in my case.

Thanx,
Amit

0 Karma

Ayn
Legend

Remove how? dedup? You could just extract the text after the timestamp as a field, or just some other specific part of the event text, and then dedup that to get rid of duplicates in your search results...

0 Karma

nowornever2
New Member

Yes Ayn,I will then remove unique repeated events from all events happened in 24 hours.

Can you help?

0 Karma

Ayn
Legend

I don't understand - do you want the text after the timestamp to be in a separate, timestamp-less event? Why? What's the use-case?

0 Karma
Get Updates on the Splunk Community!

Introduction to Splunk Observability Cloud - Building a Resilient Hybrid Cloud

Introduction to Splunk Observability Cloud - Building a Resilient Hybrid Cloud  In today’s fast-paced digital ...

Observability protocols to know about

Observability protocols define the specifications or formats for collecting, encoding, transporting, and ...

Take Your Breath Away with Splunk Risk-Based Alerting (RBA)

WATCH NOW!The Splunk Guide to Risk-Based Alerting is here to empower your SOC like never before. Join Haylee ...