topic Re: extracting mv fields in Knowledge Management

extracting mv fields

Be_JAR — Mon, 08 Apr 2024 13:12:53 GMT

Hello everyone!

I need some help creating a multivalue field.

Events can contain 1 or more fields with the following forms:

I try to explain with an example

Event1:

FICHERO_LOG1 = /any/log1/id/idca-admin/idca-admin.log

FICHERO_LOG2 = /any/log1/id/log1/any1.log

FICHERO_LOG3 = /any/log1/httpd/*

Event2:

FICHERO_LOG1 = /any/log2/id/id.log

FICHERO_LOG2 = /any/log2/logging.log

FICHERO_LOG3 = /any/log2/tree/httpd/ds/log2/*

FICHERO_LOG4 = /any/log2/id/id-batch/id-batch2.log

eventN

FICHERO_LOG1 = /any/logN/data1/activemq.log

FICHERO_LOG2 = /any/logN/id/hss2/*.system.log

………

FICHERO_LOGN = /any/path1/id/…./*…..log

The result I expect is:

For Event1

	key	values
	LOG=	/any/log1/id/idca-admin/idca-admin.log
		/any/log1/id/log1/any1.log
		/any/log1/httpd/*

for Event2:

	key	values
	LOG=	/any/log2/id/id.log
		/any/log2/logging.log
		/any/log2/tree/httpd/ds/log2/*
		/any/log2/id/idca-batch/idca-batch2.log

For event N

	key	values
	LOG=	/any/logN/data1/activemq.log
		/any/logN/id/hss2/*.system.log
		…….
		/any/path1/id/…./*…..log

I have tried with

transform.conf: [my-log] REGEX=^.*FICHERO_LOG.*\=\s*( ?<log>.*?)\s*\n MV-AD=true props.conf [extractingFields] TRANSFORM = other_transforms_stanza, my-log

But it's not working.

Any ideas or help? What steps should I follow?

Thanks

JAR

Re: extracting mv fields

richgalloway — Mon, 08 Apr 2024 19:28:18 GMT

Please explain what is meant by "it's not working". That phrase does not provide any actionable information. What are the current results and how do they differ from what you expect?

Does the "other_transforms_stanza" do anything to the data that might affect the "my-log" stanza?

Have you used regex101.com to test the REGEX?

The "^.*" construct at the beginning of the regex is meaningless. Get rid of it.

Re: extracting mv fields

marnall — Mon, 08 Apr 2024 18:18:19 GMT

Assuming your regex well extracts the fields you want, could you try this:

# transform.conf: [my-log] REGEX=FICHERO_LOG.*\=\s*( ?<log>.*?)\s*\n MV_ADD=true # props.conf [extractingFields] TRANSFORMS-ArbitraryName1 = my-log TRANSFORMS-ArbitraryName2 = other_transforms_stanza

Note that the MV ADD field is MV_ADD, not MV-AD

Re: extracting mv fields

Be_JAR — Tue, 09 Apr 2024 11:08:08 GMT

this is the result:

I would expect a LOG field to be created for each event with the different values of its log1, log2, or logn.

Regular expression works (tested on 101), and other_transforms_stanza does not apply to this field.

Re: extracting mv fields

richgalloway — Tue, 09 Apr 2024 20:20:15 GMT

I also expected the LOG field to be extracted. Were the changes to props/transforms installed on the first full Splunk instance the sees the data? Was that instance restarted? Is the screenshot showing new data (since the restart)?

Re: extracting mv fields

Be_JAR — Fri, 12 Apr 2024 09:12:07 GMT

I finally got it this way:

transform.conf: [my-log] Format = $1::$2 regex = FICHERO_LOG(\d+)\s+==s+([^=\n]+)\n MV-ADD = true

props.conf REPORT-log = my-log

thank you all for your help