Getting Data In

How to extract multiple fields using the Extract property in props.conf?

greggz
Communicator

So I'm trying to extract multiple fields using the Extract property in props.conf
The source file looks like

my.prop.1=1
my.prop.2=2
my.prop.3=3
my.prop.4=4

And I want what EACH prop becomes a field. HENCE I did not used a prefix in the regex.

[my_sourcetype]
LINE_BREAKER = ((?!))
DATETIME_CONFIG=CURRENT
TRUNCATE=100000
KV_MODE = none
EXTRACT-watt_grouping = (?m)(.+?)=(.*)

I already used multiple variations of this regex and nothing seems to work. Is there anything blocking the field extraction ?

EDIT:
At search time I provided the conrresponding pipe and it seems to extract quite nicely: | extract pairdelim=",", kvdelim="=", auto=f, limit=500, mv_add=t .. Anyone knows how to make this work in transforms.conf ? I tried it with DELIMS = "\n","=" but it breaks in a specific property prop.256 = ?'- \#&@^\!%*\:$./\;,~+=)(|}{][><` .. It extracts the previous 255 props quite nicely though

0 Karma

masonmorales
Influencer

Try this instead:

props.conf

[my_sourcetype]
LINE_BREAKER = ((?!))
DATETIME_CONFIG=CURRENT
TRUNCATE=100000
KV_MODE = none
REPORT-my_fields = multi_extract

transforms.conf

[multi_extract]
REGEX = (?m)(.+?)=(.*)
FORMAT = $1::$2
MV_ADD = true
0 Karma

greggz
Communicator

@masonmorales can you provide support to your own answer based on the feedback I gave you ?

0 Karma

masonmorales
Influencer

Are there always only 4 lines? If so you could just use 4 extract statements that are more specific.

0 Karma

greggz
Communicator

No.. they're above 400. This is the reason I don't use KV_MODE=AUTO because it has a 100 limit max extractions. The can be alphanumerical aswell or have only symbols

0 Karma

greggz
Communicator

This doesnt help.. It just clumps them all together. I want each line to become a new field with a given value.

0 Karma

greggz
Communicator

It just makes my.prop.1 = "1 my.prop.2 = 2 my.prop.3 = 3 ..."

0 Karma

greggz
Communicator

Thanks, let me try it out

0 Karma

somesoni2
Revered Legend

Your source, is it one line per event or all those properties appear in same event? What are the fields that are automatically extracted by Splunk?

0 Karma

greggz
Communicator

None, because I made it KV_MODE=none. No, All lines come in the same event. I purposedly made it so with the LINE_BREAKER = ((?!))

0 Karma

greggz
Communicator

@somesoni2 ♦ Do you know how this should be done ?

0 Karma
Get Updates on the Splunk Community!

Monitoring MariaDB and MySQL

In a previous post, we explored monitoring PostgreSQL and general best practices around which metrics to ...

Financial Services Industry Use Cases, ITSI Best Practices, and More New Articles ...

Splunk Lantern is a Splunk customer success center that provides advice from Splunk experts on valuable data ...

Splunk Federated Analytics for Amazon Security Lake

Thursday, November 21, 2024  |  11AM PT / 2PM ET Register Now Join our session to see the technical ...