how to Configure positional timestamp extraction in log using RegEx?

[mallempatisreed]

hi All,

Am trying to extract the time stamp inside event as index time. We have similar sourcetype of logs from 4 different indexes

Apr 19 09:21:12 XYZADMXYZAB3P04 XYZADMXYZAB3P04 [customer] [426426|TaskExecutor-master-426426-ProcessTask [8797404726198]] [2018-04-19 09:21:11,929] [not present] [admin] [true]: Customer [[email protected]] is created/updated
Apr 25 15:00:44 XYZADMXYZAB3P04 XYZADMXYZAB3P04 [customer] [139468|TaskExecutor-master-139468-ProcessTask [8797864231862]] [2018-04-25 15:00:41,004] [not present] [admin] [true]: Customer [[email protected]] is created/updated
Apr 4 09:52:28 XYZECMXYZAB1P43 XYZECMXYZAB1P43 [customer] [103920|ajp-bio-8010-exec-158] [2018-04-04 09:52:21,843] [192.145.12.4] [[email protected]] [true]: Customer [[email protected]] is created/updated
Apr 4 09:52:28 XYZECMXYZAB1P43 XYZECMXYZAB1P43 [authentication] [103920|ajp-bio-8010-exec-158] [2018-04-04 09:52:21,876] [192.145.12.4] [[email protected]] [true]: user [[email protected]] successfully authenticated
Apr 9 12:41:52 XYZBUSXYZAB3P01 XYZBUSXYZAB3P01 [employee] [200061|hybrisHTTP8] [2018-04-09 12:41:48,609] [10.44.189.72] [anonymous] [true]: Employee [tester] is created/updated

Apr 21 02:55:46 ABCPUBXYZAB56 ABCPUBXYZAB56 2018-04-21 02:55:39.800 INFO [com.xyxf.auth.core.XYLoginHookAuthenticationHandler] Activated XYZ authentication feedback handler wrap handler enabled is true
Apr 12 08:23:06 ABCPUBXYZAB47 ABCPUBXYZAB47 2018-04-12 08:23:00.401 INFO [com.xyzf.auth.core.XYLoginHookAuthenticationHandler] 10.66.101.22 admin failed

In the above logs how to extract the second timestamp as indextime. can someone help me with RegEx.

Thanks,
Sree

[TISKAR]

@mallempatisreedhar , can you try this please:

| rex "^.*(?=(\d{4}-\d+-\d+\s\d+:\d+:\d+[,.]\d+))(?<time>\d{4}-\d+-\d+\s\d+:\d+:\d+[,.]\d+)"

if you want extract the time at parsing using sourcetype stanza props.conf:

TIME_FORMAT=%Y-%m-%d %H:%M:%S
TIME_PREFIX=^.*(?=(\d{4}-\d+-\d+\s\d+:\d+:\d+[,.]\d+))

[horsefez]

Hi Mallempatisreeedhar,

I wrote a regex that should help you extract the timestamps you want.

Click here to see the regex. Also regex101 is a great site for developing those regexes.

https://regex101.com/r/62Pfpn/1

If you want a SPL example:
| rex field=_raw "\[?(?<time>20\d\d-\d\d-\d\d\s*\d\d:\d\d:\d\d(?:\,|\.)\d+)\[?"