topic Re: Regex Help in Splunk Search

Regex Help

bschaap — Tue, 29 Sep 2020 20:17:24 GMT

I'm trying to parse out the exception type and exception message from the DB Connect dbx_server logs. I'm having some difficulty with this and have tried regex101.com and am able to come up with a regex that parses it but cannot get it to work using the rex command in Splunk. In the sample event below, I am trying to parse out "java.io.IOException" and "HTTP Error 503: Service Unavailable".

From regex101 for the exception message -
Exception: (?P.+? at)

Event -
2018-07-05 16:21:05.528 -0400 [QuartzScheduler_Worker-21] ERROR c.s.d.s.task.listeners.RecordWriterMetricsListener - action=unable_to_write_batch java.io.IOException: HTTP Error 503: Service Unavailable at com.splunk.dbx.server.dbinput.recordwriter.HttpEventCollector.uploadEventBatch(HttpEventCollector.java:112) at com.splunk.dbx.server.dbinput.recordwriter.HttpEventCollector.uploadEvents(HttpEventCollector.java:89) at com.splunk.dbx.server.dbinput.recordwriter.HecEventWriter.writeRecords(HecEventWriter.java:36) at org.easybatch.core.job.BatchJob.writeBatch(BatchJob.java:203) at org.easybatch.core.job.BatchJob.call(BatchJob.java:79) at org.easybatch.extensions.quartz.Job.execute(Job.java:59) at org.quartz.core.JobRunShell.run(JobRunShell.java:202) at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:573) 2018-07-05 16:21:05.528 -0400 [QuartzScheduler_Worker-21] ERROR c.s.d.s.dbinput.recordwriter.CheckpointUpdater - action=skip_checkpoint_update_batch_writing_failed java.io.IOException: HTTP Error 503: Service Unavailable at com.splunk.dbx.server.dbinput.recordwriter.HttpEventCollector.uploadEventBatch(HttpEventCollector.java:112) at com.splunk.dbx.server.dbinput.recordwriter.HttpEventCollector.uploadEvents(HttpEventCollector.java:89) at com.splunk.dbx.server.dbinput.recordwriter.HecEventWriter.writeRecords(HecEventWriter.java:36) at org.easybatch.core.job.BatchJob.writeBatch(BatchJob.java:203) at org.easybatch.core.job.BatchJob.call(BatchJob.java:79) at org.easybatch.extensions.quartz.Job.execute(Job.java:59) at org.quartz.core.JobRunShell.run(JobRunShell.java:202) at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:573)

Re: Regex Help

somesoni2 — Thu, 05 Jul 2018 21:14:27 GMT

Give this a try

Your base search
| rex "^(\S+\s+){8}(?P<Exception_Type>[^\:]+):\s*(?<Exception_Message>.+? at)"

Re: Regex Help

PowerPacked — Thu, 05 Jul 2018 21:27:03 GMT

Hi @bschaap

Take a look at this

if you want to do it in rex command:

java.io.IOException:(?P<fieldname>.*?) at

Thanks

Re: Regex Help

bschaap — Tue, 29 Sep 2020 20:17:30 GMT

Thanks. I tried this but unfortunately it's not parsing the field.

Results
"_time",fieldname,"Exception_Type","Exception_Message"
"2018-07-05T17:47:08.000-0400",,,

Search
| makeresults
| eval raw = "2018-07-05 16:21:05.528 -0400 [QuartzScheduler_Worker-21] ERROR c.s.d.s.task.listeners.RecordWriterMetricsListener - action=unable_to_write_batch java.io.IOException: HTTP Error 503: Service Unavailable at com.splunk.dbx.server.dbinput.recordwriter.HttpEventCollector.uploadEventBatch(HttpEventCollector.java:112) at com.splunk.dbx.server.dbinput.recordwriter.HttpEventCollector.uploadEvents(HttpEventCollector.java:89) at com.splunk.dbx.server.dbinput.recordwriter.HecEventWriter.writeRecords(HecEventWriter.java:36) at org.easybatch.core.job.BatchJob.writeBatch(BatchJob.java:203) at org.easybatch.core.job.BatchJob.call(BatchJob.java:79) at org.easybatch.extensions.quartz.Job.execute(Job.java:59) at org.quartz.core.JobRunShell.run(JobRunShell.java:202) at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:573) 2018-07-05 16:21:05.528 -0400 [QuartzScheduler_Worker-21] ERROR c.s.d.s.dbinput.recordwriter.CheckpointUpdater - action=skip_checkpoint_update_batch_writing_failed java.io.IOException: HTTP Error 503: Service Unavailable at com.splunk.dbx.server.dbinput.recordwriter.HttpEventCollector.uploadEventBatch(HttpEventCollector.java:112) at com.splunk.dbx.server.dbinput.recordwriter.HttpEventCollector.uploadEvents(HttpEventCollector.java:89) at com.splunk.dbx.server.dbinput.recordwriter.HecEventWriter.writeRecords(HecEventWriter.java:36) at org.easybatch.core.job.BatchJob.writeBatch(BatchJob.java:203) at org.easybatch.core.job.BatchJob.call(BatchJob.java:79) at org.easybatch.extensions.quartz.Job.execute(Job.java:59) at org.quartz.core.JobRunShell.run(JobRunShell.java:202) at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:573)"
| rex "java.io.IOException:(?P.?) at"
| rex "^(\S+\s+){8}(?P[^:]+):\s(?.+? at)"
| table _time fieldname Exception_Type Exception_Message

Re: Regex Help

bschaap — Thu, 05 Jul 2018 21:51:45 GMT

Thanks. I tried this but unfortunately it's not parsing the field. See the previous comment for my search.

Re: Regex Help

PowerPacked — Thu, 05 Jul 2018 22:06:44 GMT

Adding new answer, as i cant post image in comments.

Thanks