Splunk Search
Highlighted

How to configure multivalue field extraction?

Path Finder

Hello,

I cannot configure multivalue field extraction. I have a following event. the last 4 lines Time Stamp and Message shall be extracted as separate values togather with value following the FROM: section on the first line. I used props.conf. and Transforms.conf (MV_ADD) however no use.

From: "Rnvr" <r@cou.ge>
Subject:  Control Center System Event
Date: Fri, 15 Jun 2018 18:14:07 +0400
Message-ID: <CE4FFC373CC843A4AFE30E855AE5839F@court.gov.ge>
Return-Path: r@cou.ge
Received: from mail.cou.ge (LHLO mail.cou.ge) (192.168.222.10) by
 mail.cou.ge with LMTP; Fri, 15 Jun 2018 18:13:58 +0400 (GET)
Received: from localhost (localhost [])
    by mail.court.ge (Postfix) with ESMTP id 75C1519E007B
    for <asupport@cou.ge>; Fri, 15 Jun 2018 18:13:58 +0400 (+04)
[2018-Jun-15 06:04:42 PM (GET)] Hardware event occurred (The controller write policy has been changed to Write Back.) on server 

[2018-Jun-15 06:04:43 PM (GET)] Hardware event occurred (The virtual disk cache policy has changed.) on server 

[2018-Jun-15 06:04:44 PM (GET)] Hardware event occurred (The virtual disk cache policy has changed.) on server 

[2018-Jun-15 06:13:16 PM (GET)] Digital input 'Digital Input 1' deactivated.
0 Karma
Highlighted

Re: How to configure multivalue field extraction?

Legend

Please include relevant sections from props.conf and transforms.conf, and elaborate on how things are not working. Are you getting any relevant fields at all?

0 Karma
Highlighted

Re: How to configure multivalue field extraction?

Path Finder

Hello, yes I'm getting results but it is not extracting all the values in 1 event. here are my conf. file sections.

Transforms:

[date]
REGEX= (?<=\[)(?P<date>\d{4}\-\w{3}\-\d{2}\s\d{2}\:\d{2}\:\d{2}\s\w{2})(?=\s\(\w{3})
MV_ADD = true


[message]
REGEX=(?<=\[\d{4}\-\w{3}\-\d{2}\s\d{2}\:\d{2}\:\d{2}\s\w{2}\s\(\w{3}\)\]\s)(?P<message>\w.*\X)
MV_ADD = true


Props:

[mysourcetype]
DATETIME_CONFIG = 
LINE_BREAKER = ()\wrom:\s"
NO_BINARY_CHECK = true
category = Custom
disabled = false
pulldown_type = true
REPORT-date = date
REPORT-message= message
0 Karma
Highlighted

Re: How to configure multivalue field extraction?

Ultra Champion

Well, the regular expressions seem to be OK. Have you deployed this on the Search Head(s)?

The transforms namings are a bit short and generic, you might want to check with btool if those are unique.

0 Karma
Highlighted

Re: How to configure multivalue field extraction?

Path Finder

Frank yes it is a SH. as for the naming yes they are unique, the problem is only with the message extraction.

in message field extracted results shows exactly like this - Hardware event occurred (The controller write policy has been changed to Write Back.) on server RUSTAVI-VIDEO. [2018-Jun-15 06:04:43 PM (GET)] Hardware event occurred (The virtual disk cache policy has changed.) on server RUSTAVI-VIDEO. [2018-Jun-15 06:04:44 PM (GET)] Hardware event occurred (The virtual disk cache policy has changed.) on server RUSTAVI-VIDEO. [2018-Jun-15 06:13:16 PM (GET)] Digital input 'Digital Input 1' deactivated.

even though my regex says to ignore the beginning (date) it still appears to be in results and moreover, it does not separate messages as a separate values, it shows it as a one big string.

0 Karma
Highlighted

Re: How to configure multivalue field extraction?

Ultra Champion

Looks like the extraction of the first message doesn't stop at the end of the line and just captures the whole last 4 lines as 1 message entry. You may need to adjust your regex slightly to ensure it only captures a single message at a time.

For example: (?<=\[\d{4}\-\w{3}\-\d{2}\s\d{2}\:\d{2}\:\d{2}\s\w{2}\s\(\w{3}\)\]\s)(?P<message>\w[^\r\n]+)

View solution in original post

0 Karma
Highlighted

Re: How to configure multivalue field extraction?

Path Finder

thanks it worked for me.

0 Karma