topic How to configure multivalue field extraction? in Splunk Search

How to configure multivalue field extraction?

ninisimonishvil — Tue, 26 Jun 2018 06:16:58 GMT

Hello,

I cannot configure multivalue field extraction. I have a following event. the last 4 lines Time Stamp and Message shall be extracted as separate values togather with value following the FROM: section on the first line. I used props.conf. and Transforms.conf (MV_ADD) however no use.

From: "Rnvr" <r@cou.ge>
Subject:  Control Center System Event
Date: Fri, 15 Jun 2018 18:14:07 +0400
Message-ID: <CE4FFC373CC843A4AFE30E855AE5839F@court.gov.ge>
[2018-Jun-15 06:04:42 PM (GET)] Hardware event occurred (The controller write policy has been changed to Write Back.) on server 

[2018-Jun-15 06:04:43 PM (GET)] Hardware event occurred (The virtual disk cache policy has changed.) on server 

[2018-Jun-15 06:04:44 PM (GET)] Hardware event occurred (The virtual disk cache policy has changed.) on server 

[2018-Jun-15 06:13:16 PM (GET)] Digital input 'Digital Input 1' deactivated.

Re: How to configure multivalue field extraction?

Ayn — Tue, 26 Jun 2018 07:20:35 GMT

Please include relevant sections from props.conf and transforms.conf, and elaborate on how things are not working. Are you getting any relevant fields at all?

Re: How to configure multivalue field extraction?

ninisimonishvil — Tue, 26 Jun 2018 07:30:21 GMT

Hello, yes I'm getting results but it is not extracting all the values in 1 event. here are my conf. file sections.

Transforms:

[date]
REGEX= (?<=\[)(?P<date>\d{4}\-\w{3}\-\d{2}\s\d{2}\:\d{2}\:\d{2}\s\w{2})(?=\s\(\w{3})
MV_ADD = true


[message]
REGEX=(?<=\[\d{4}\-\w{3}\-\d{2}\s\d{2}\:\d{2}\:\d{2}\s\w{2}\s\(\w{3}\)\]\s)(?P<message>\w.*\X)
MV_ADD = true


Props:

[mysourcetype]
DATETIME_CONFIG = 
LINE_BREAKER = ()\wrom:\s"
NO_BINARY_CHECK = true
category = Custom
disabled = false
pulldown_type = true
REPORT-date = date
REPORT-message= message

Re: How to configure multivalue field extraction?

FrankVl — Tue, 26 Jun 2018 08:02:06 GMT

Well, the regular expressions seem to be OK. Have you deployed this on the Search Head(s)?

The transforms namings are a bit short and generic, you might want to check with btool if those are unique.

Re: How to configure multivalue field extraction?

ninisimonishvil — Tue, 26 Jun 2018 08:10:49 GMT

Frank yes it is a SH. as for the naming yes they are unique, the problem is only with the message extraction.

in message field extracted results shows exactly like this - Hardware event occurred (The controller write policy has been changed to Write Back.) on server RUSTAVI-VIDEO. [2018-Jun-15 06:04:43 PM (GET)] Hardware event occurred (The virtual disk cache policy has changed.) on server RUSTAVI-VIDEO. [2018-Jun-15 06:04:44 PM (GET)] Hardware event occurred (The virtual disk cache policy has changed.) on server RUSTAVI-VIDEO. [2018-Jun-15 06:13:16 PM (GET)] Digital input 'Digital Input 1' deactivated.

even though my regex says to ignore the beginning (date) it still appears to be in results and moreover, it does not separate messages as a separate values, it shows it as a one big string.

Re: How to configure multivalue field extraction?

FrankVl — Tue, 26 Jun 2018 08:19:57 GMT

Looks like the extraction of the first message doesn't stop at the end of the line and just captures the whole last 4 lines as 1 message entry. You may need to adjust your regex slightly to ensure it only captures a single message at a time.

For example: (?<=\[\d{4}\-\w{3}\-\d{2}\s\d{2}\:\d{2}\:\d{2}\s\w{2}\s\(\w{3}\)\]\s)(?P<message>\w[^\r\n]+)

Re: How to configure multivalue field extraction?

ninisimonishvil — Tue, 26 Jun 2018 12:05:35 GMT

thanks it worked for me.