topic Re: Transfer decimal value into ASCII code in text string. in Splunk Search

How to transfer decimal value into ASCII code in text string?

Jouman — Fri, 05 May 2023 11:06:03 GMT

Hi all,

I have a field named as item_description which is an array of decimal value, which represents the description of each item.

I hope to transfer each value in item_description into text string for each item.

Original data:

| makeresults | eval item_name = "Name_1,Name_2,Name_3,Name_4,Name_5", item_description = "65_66_67,68_69_70,71_72_73,74_75_76,77_78_79" | makemv delim="," item_name | makemv delim="," item_description | eval mv_zipped=mvzip(item_name,item_description) | mvexpand mv_zipped | rex field=mv_zipped "(?P<ITEM_NAME>.*),(?P<ITEM_DESP>.*)" | makemv delim="_" ITEM_DESP | table _time ITEM_NAME ITEM_DESP

Although the purpose can be fulfilled by the following code.

Output:

_time	ITEM_NAME	ITEM_DESP_join
XXX	Name_1	ABC
YYY	Name_2	DEF
ZZZ	Name_3	GHI
000	Name_4	JKL
111	Name_5	MNO

If the item_description becomes very long(ex. lengh=50) and lots of items (ex. 50 items), the mvexpand command can't work properly with the output message below.
Error message:
command.mvexpand: output will be truncated at 28200 results due to excessive memory usage. Memory threshold of 500MB as configured in limits.conf / [mvexpand] / max_mem_usage_mb has been reached.

Is there any other way to transfer decimal value into ASCII and make the output as a string without using mvexpand command?

Thank you very much.

Re: Transfer decimal value into ASCII code in text string.

yuanliu — Fri, 05 May 2023 08:38:49 GMT

Do you mean mvmap?

ITEM_NAME	ITEM_DESP	ITEM_DESP_char
Name_1	65 66 67	A B C
Name_2	68 69 70	D E F
Name_3	71 72 73	G H I
Name_4	74 75 76	J K L
Name_5	77 78 79	M N O

Re: Transfer decimal value into ASCII code in text string.

Jouman — Sat, 06 May 2023 05:10:56 GMT

Hi Yuan,

Thank you for the reply.

The key issue in my code is, if the items or the item description become very long, there will be warning messages for mvexpand due to too many memory usage.

Is there any other method to avoid using mvexpand while converting decimal values into text array?

Thank you.

Re: Transfer decimal value into ASCII code in text string.

yuanliu — Sat, 06 May 2023 05:26:55 GMT

Have you tried my mvmap example? It operates "locally" so to speak, processing one field in one event at a time. mvexpand creates a ton of extra events (rows) that carry everything from the original row, therefore demands a lot more memory.

Re: Transfer decimal value into ASCII code in text string.

Jouman — Sat, 06 May 2023 19:15:57 GMT

Hi @yuanliu ,

Sorry for misunderstanding your reply at first. I saw mvexpand and I thought mvexpand() is part of the solution.

Thank you to introduce mvmap().
This command solves the issue indeed.

Thank you very much!