Getting Data In

How to extract multiple fields using the Extract property in props.conf?

greggz
Communicator

So I'm trying to extract multiple fields using the Extract property in props.conf
The source file looks like

my.prop.1=1
my.prop.2=2
my.prop.3=3
my.prop.4=4

And I want what EACH prop becomes a field. HENCE I did not used a prefix in the regex.

[my_sourcetype]
LINE_BREAKER = ((?!))
DATETIME_CONFIG=CURRENT
TRUNCATE=100000
KV_MODE = none
EXTRACT-watt_grouping = (?m)(.+?)=(.*)

I already used multiple variations of this regex and nothing seems to work. Is there anything blocking the field extraction ?

EDIT:
At search time I provided the conrresponding pipe and it seems to extract quite nicely: | extract pairdelim=",", kvdelim="=", auto=f, limit=500, mv_add=t .. Anyone knows how to make this work in transforms.conf ? I tried it with DELIMS = "\n","=" but it breaks in a specific property prop.256 = ?'- \#&@^\!%*\:$./\;,~+=)(|}{][><` .. It extracts the previous 255 props quite nicely though

0 Karma

masonmorales
Influencer

Try this instead:

props.conf

[my_sourcetype]
LINE_BREAKER = ((?!))
DATETIME_CONFIG=CURRENT
TRUNCATE=100000
KV_MODE = none
REPORT-my_fields = multi_extract

transforms.conf

[multi_extract]
REGEX = (?m)(.+?)=(.*)
FORMAT = $1::$2
MV_ADD = true
0 Karma

greggz
Communicator

@masonmorales can you provide support to your own answer based on the feedback I gave you ?

0 Karma

masonmorales
Influencer

Are there always only 4 lines? If so you could just use 4 extract statements that are more specific.

0 Karma

greggz
Communicator

No.. they're above 400. This is the reason I don't use KV_MODE=AUTO because it has a 100 limit max extractions. The can be alphanumerical aswell or have only symbols

0 Karma

greggz
Communicator

This doesnt help.. It just clumps them all together. I want each line to become a new field with a given value.

0 Karma

greggz
Communicator

It just makes my.prop.1 = "1 my.prop.2 = 2 my.prop.3 = 3 ..."

0 Karma

greggz
Communicator

Thanks, let me try it out

0 Karma

somesoni2
Revered Legend

Your source, is it one line per event or all those properties appear in same event? What are the fields that are automatically extracted by Splunk?

0 Karma

greggz
Communicator

None, because I made it KV_MODE=none. No, All lines come in the same event. I purposedly made it so with the LINE_BREAKER = ((?!))

0 Karma

greggz
Communicator

@somesoni2 ♦ Do you know how this should be done ?

0 Karma
Get Updates on the Splunk Community!

Extending Observability Content to Splunk Cloud

Watch Now!   In this Extending Observability Content to Splunk Cloud Tech Talk, you'll see how to leverage ...

More Control Over Your Monitoring Costs with Archived Metrics!

What if there was a way you could keep all the metrics data you need while saving on storage costs?This is now ...

New in Observability Cloud - Explicit Bucket Histograms

Splunk introduces native support for histograms as a metric data type within Observability Cloud with Explicit ...