props.conf help

a212830 — Mon, 28 Sep 2020 17:09:02 GMT

Hi,

I have a multi-line feed that I'm having problems with - hoping someone can help me. Here's my props.conf:

MAX_TIMESTAMP_LOOKAHEAD = 30
NO_BINARY_CHECK = 1
TIME_FORMAT = %Y-%m-%d %H:%M:%S,%3N
TIME_PREFIX = ^
TRUNCATE = 999999
SHOULD_LINEMERGE = false
LINE_BREAKER = ([\r\n]+)\d{4}\s\w{2}\s\d{2}

And, here's some sample data. The break is not occurring on the lines properly. Pretty sure my LINE_BREAKER is wrong. On a side note, if I just let splunk do the work, everything is wonderful, but this is not how we want to eng our systems. Is there any way to tell what splunk is applying for timestamps when you don't specify a props.conf?

2014-07-23 02:44:46,913 INFO [com.mpi.saml.plugin.SamlTokenRepositoryCleanupTask] Deleting the expired tokens from SAML repository
2014-07-23 02:46:21,620 INFO [com.media3k.escene.util.background.SsoNonceCleaner] DEFAULT.SsoNonceCleaner executing at Wed Jul 23 02:46:21 EDT 2014
2014-07-23 02:59:46,914 INFO [com.mpi.saml.plugin.SamlTokenRepositoryCleanupTask] Deleting the expired tokens from SAML repository
2014-07-23 03:01:21,693 INFO [com.media3k.escene.util.background.SsoNonceCleaner] DEFAULT.SsoNonceCleaner executing at Wed Jul 23 03:01:21 EDT 2014
2014-07-23 03:06:21,621 INFO [com.media3k.escene.util.background.SsoNonceCleaner] DEFAULT.SsoNonceCleaner executing at Wed Jul 23 03:06:21 EDT 2014
2014-07-23 03:14:15,379 ERROR [com.media3k.escene.util.background.TempFileCleaner] Unable to find ingesting Programs to activate.
java.lang.NullPointerException: null
at com.qumu.prg.builder.ProgramAssociationService.findProgramContentAssociation(ProgramAssociationService.java:168) ~[mpi-bcast.jar:na]
at com.media3k.escene.util.background.TempFileCleaner.processImage(TempFileCleaner.java:115) [adminportal.jar:na]
at com.media3k.escene.util.background.TempFileCleaner.access$200(TempFileCleaner.java:39) [adminportal.jar:na]
at com.media3k.escene.util.background.TempFileCleaner$1.call(TempFileCleaner.java:70) [adminportal.jar:na]
at com.media3k.escene.util.background.TempFileCleaner$1.call(TempFileCleaner.java:57) [adminportal.jar:na]
at com.mpi.framework.threadcontext.ThreadContexts$ContextCallableImpl.call(ThreadContexts.java:141) [foundation.jar:na]
at com.media3k.escene.util.background.TempFileCleaner.finalizeEgcPrograms(TempFileCleaner.java:57) [adminportal.jar:na]
at com.media3k.escene.util.background.TempFileCleaner.executeInternal(TempFileCleaner.java:51) [adminportal.jar:na]
at org.springframework.scheduling.quartz.QuartzJobBean.execute(QuartzJobBean.java:113) [spring-context-support.jar:3.2.1.RELEASE]
at org.quartz.core.JobRunShell.run(JobRunShell.java:216) [quartz.jar:na]
at com.media3k.escene.util.background.TaskExecutionServiceThreadPool$QuartzTask.call(TaskExecutionServiceThreadPool.java:93) [mpi-lodgepole.jar:na]
at com.media3k.escene.util.background.TaskExecutionServiceThreadPool$QuartzTask.call(TaskExecutionServiceThreadPool.java:82) [mpi-lodgepole.jar:na]
at java.util.concurrent.FutureTask.run(FutureTask.java:262) [na:1.7.0_65]
at com.mpi.framework.task.impl.juc.JUCThreadPool$InternalFutureTask.run(JUCThreadPool.java:328) [foundation.jar:na]
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) [na:1.7.0_65]
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) [na:1.7.0_65]
at java.lang.Thread.run(Thread.java:745) [na:1.7.0_65]
2014-07-23 03:14:15,379 INFO [com.media3k.escene.util.background.TempFileCleaner] CentralConfig.filerTempDir not set or does not exist; using D:\Temp for file upload directory.
2014-07-23 03:14:15,380 INFO [com.media3k.escene.util.background.TempFileCleaner] Cleaning up the temp folder: D:\Temp
2014-07-23 03:14:46,915 INFO [com.mpi.saml.plugin.SamlTokenRepositoryCleanupTask] Deleting the expired tokens from SAML repository
2014-07-23 03:21:21,634 INFO [com.media3k.escene.util.background.SsoNonceCleaner] DEFAULT.SsoNonceCleaner executing at Wed Jul 23 03:21:21 EDT 2014

Re: props.conf help

a212830 — Mon, 28 Sep 2020 17:09:04 GMT

Oh, maybe...?

MAX_TIMESTAMP_LOOKAHEAD = 30
NO_BINARY_CHECK = 1
TIME_FORMAT = %Y-%m-%d %H:%M:%S,%3N
TIME_PREFIX = ^
TRUNCATE = 999999
SHOULD_LINEMERGE = false
LINE_BREAKER = ([\r\n]+)\d{4}-\d{2}-\d{2}

Re: props.conf help

richgalloway — Wed, 23 Jul 2014 13:42:19 GMT

Your second LINE_BREAKER string should do it. If not, try '([\r\n]+)(\d+)-(\d+)-(\d+)'

Re: props.conf help

linu1988 — Wed, 23 Jul 2014 13:42:53 GMT

Hello,
The below would actually be appropriate in your case.

NO_BINARY_CHECK = 1 TIME_FORMAT = %Y-%m-%d %H:%M:%S,%3N TRUNCATE = 0 SHOULD_LINEMERGE = false BREAK_ONLY_BEFORE_DATE = true

NO_BINARY_CHECK = 1 TIME_FORMAT = %Y-%m-%d %H:%M:%S,%3N TRUNCATE = 0 SHOULD_LINEMERGE = false BREAK_ONLY_BEFORE =\d{4}-\d{2}-\d{2} \d{2}:\d{2}:\d{2}

Thanks,
L

topic Re: props.conf help in Getting Data In

props.conf help

Re: props.conf help

Re: props.conf help

Re: props.conf help