Splunk Search

EVAL with field extracted from props.conf

johnmvang
Path Finder

Hello Everyone,

I'm having an issue where I cannot use EVAL in search or in the props.conf for a field that has been previously generated from the props.conf file.

Context:
I have a file like this

Heading_Line-0
D3012010900ABC803M R00011122 00000000000000012340000020621 CHK

D3012010900ABC803M R00112233 00000000000000001230000001435 CHK

D3012010900ABC809K R33222110 00000000000000043210000000100 CHK

D3012010900ABC807T R22200111 00000000000000054320000035663 CHK

I have already (successfully) extracted with the props.conf/transforms.conf a multi-value field called "check_amount" from the last 10 digits in the big set of numbers starting with all zeros

my props.conf reads:

[checks_source]
REPORT-mvfields = my_check_amount

my transforms.conf reads:

[my_check_amount]
REGEX = (?m)^\w+\s+\w+\s+\w{19}(?\d{10})
MV_ADD = true

0000000000000001234 0000020621
0000000000000000123 0000001435
0000000000000004321 0000000100
0000000000000005432 0000035663

Now when i run my search:

sourcetype=checks_source | eval check_amount = round(check_amount / 100, 2)

The check_amount field disappears. But I need to get a decimal point inbetween the last 2 digits and get rid of all the preceding zeroes.

my desired finished result would be to get the dollar amount in human readable format:

0000020621 = 206.21
0000001435 = 14.35
0000000100 = 1.00
0000035663 = 356.63

Any help is greatly appreciated.

Thanks,

John

0 Karma
1 Solution

johnmvang
Path Finder

Micahkemp answered this question. However he posted a comment, so i could not mark it, so i'm just adding the details and marking it as answered.

With my dilemma use MVEXPAND:

| mvexpand check_amount |eval fixed_value = round(check_amount / 100, 2)

or index the data as separate lines and then use the |EVAL in a search or in the props to have the work performed automatically.

Thanks,

John

View solution in original post

0 Karma

johnmvang
Path Finder

Micahkemp answered this question. However he posted a comment, so i could not mark it, so i'm just adding the details and marking it as answered.

With my dilemma use MVEXPAND:

| mvexpand check_amount |eval fixed_value = round(check_amount / 100, 2)

or index the data as separate lines and then use the |EVAL in a search or in the props to have the work performed automatically.

Thanks,

John

0 Karma

woodcock
Esteemed Legend

Be sure to upvote his comments.

0 Karma

micahkemp
Champion

Is there any reason not to split those into multiple events? If not, you could do:

| mvexpand check_amount |eval fixed_value = round(check_amount / 100, 2)

Which would result in one event per check.

Edit: converted from comment to answer.

0 Karma

johnmvang
Path Finder

Should i remove my existing props/transforms that is already doing the multi-value extraction prior to using mvexpand?

0 Karma

micahkemp
Champion

No, you need the values to be extracted before using mvexpand.

However, you might want to reconsider how your data is indexed, if it makes more sense for each line to be a separate event (which from an outsider point of view makes the most sense to me).

johnmvang
Path Finder

I might be be able to do just that. I know indexing this file and breaking the file at each line will make life easier, I just had other data in the heading of the file that may be needed... I don't know yet.

Thanks man. I'll mark this as answered.

0 Karma

micahkemp
Champion

Do you still have this issue if you run the search in verbose mode, or changing the name of the field resulting from eval?

0 Karma

johnmvang
Path Finder

I still have this issue in verbose mode or if i change the field name with EVAL.

If i did "|eval fixed_value = round(check_amount / 100, 2)" the "fixed_value" field never shows up.

0 Karma

micahkemp
Champion

And you see the check_amount field populated using that last search?

0 Karma

johnmvang
Path Finder

yes, the check_amount field is still populated with the old set of values 0000020621 and etc. but the new fixed_value field is not there and i have no modified values: 206.21, etc.

0 Karma

micahkemp
Champion

Is check_amount a multi-value field, as indicated by MV_ADD=true? I think that may break your eval.

johnmvang
Path Finder

Yes that is a multi-value field and I believe this is the problem. However, i'm looking for a solution to see if there is any way around it to get the desired results.

0 Karma
Get Updates on the Splunk Community!

Routing Data to Different Splunk Indexes in the OpenTelemetry Collector

This blog post is part of an ongoing series on OpenTelemetry. The OpenTelemetry project is the second largest ...

Getting Started with AIOps: Event Correlation Basics and Alert Storm Detection in ...

Getting Started with AIOps:Event Correlation Basics and Alert Storm Detection in Splunk IT Service ...

Register to Attend BSides SPL 2022 - It's all Happening October 18!

Join like-minded individuals for technical sessions on everything Splunk!  This is a community-led and run ...