EVAL with field extracted from props.conf

[johnmvang]

Hello Everyone,

I'm having an issue where I cannot use EVAL in search or in the props.conf for a field that has been previously generated from the props.conf file.

Context:
I have a file like this

Heading_Line-0
D3012010900ABC803M R00011122 00000000000000012340000020621 CHK

D3012010900ABC803M R00112233 00000000000000001230000001435 CHK

D3012010900ABC809K R33222110 00000000000000043210000000100 CHK

D3012010900ABC807T R22200111 00000000000000054320000035663 CHK

I have already (successfully) extracted with the props.conf/transforms.conf a multi-value field called "check_amount" from the last 10 digits in the big set of numbers starting with all zeros

my props.conf reads:

[checks_source]
REPORT-mvfields = my_check_amount

my transforms.conf reads:

[my_check_amount]
REGEX = (?m)^\w+\s+\w+\s+\w{19}(?\d{10})
MV_ADD = true

0000000000000001234 0000020621
0000000000000000123 0000001435
0000000000000004321 0000000100
0000000000000005432 0000035663

Now when i run my search:

sourcetype=checks_source | eval check_amount = round(check_amount / 100, 2)

The check_amount field disappears. But I need to get a decimal point inbetween the last 2 digits and get rid of all the preceding zeroes.

my desired finished result would be to get the dollar amount in human readable format:

0000020621 = 206.21
0000001435 = 14.35
0000000100 = 1.00
0000035663 = 356.63

Any help is greatly appreciated.

Thanks,

John

[johnmvang · Accepted solution]

Micahkemp answered this question. However he posted a comment, so i could not mark it, so i'm just adding the details and marking it as answered.

With my dilemma use MVEXPAND:

| mvexpand check_amount |eval fixed_value = round(check_amount / 100, 2)

or index the data as separate lines and then use the |EVAL in a search or in the props to have the work performed automatically.

Thanks,

John

[woodcock]

Be sure to upvote his comments.

[micahkemp]

Is there any reason not to split those into multiple events? If not, you could do:

| mvexpand check_amount |eval fixed_value = round(check_amount / 100, 2)

Which would result in one event per check.

Edit: converted from comment to answer.

[johnmvang]

Should i remove my existing props/transforms that is already doing the multi-value extraction prior to using mvexpand?

[micahkemp]

No, you need the values to be extracted before using mvexpand.

However, you might want to reconsider how your data is indexed, if it makes more sense for each line to be a separate event (which from an outsider point of view makes the most sense to me).

[johnmvang]

I might be be able to do just that. I know indexing this file and breaking the file at each line will make life easier, I just had other data in the heading of the file that may be needed... I don't know yet.

Thanks man. I'll mark this as answered.

[micahkemp]

Do you still have this issue if you run the search in verbose mode, or changing the name of the field resulting from eval?

[johnmvang]

I still have this issue in verbose mode or if i change the field name with EVAL.

If i did "|eval fixed_value = round(check_amount / 100, 2)" the "fixed_value" field never shows up.

[micahkemp]

And you see the check_amount field populated using that last search?

[johnmvang]

yes, the check_amount field is still populated with the old set of values 0000020621 and etc. but the new fixed_value field is not there and i have no modified values: 206.21, etc.

[micahkemp]

Is check_amount a multi-value field, as indicated by MV_ADD=true? I think that may break your eval.

[johnmvang]

Yes that is a multi-value field and I believe this is the problem. However, i'm looking for a solution to see if there is any way around it to get the desired results.