How to transfer decimal value into ASCII code in text string?

[Jouman]

Hi all, 

I have a field named as item_description which is an array of decimal value, which represents the description of each item.

I hope to transfer each value in item_description into text string for each item.

Original data:

| makeresults
| eval item_name = "Name_1,Name_2,Name_3,Name_4,Name_5", item_description = "65_66_67,68_69_70,71_72_73,74_75_76,77_78_79"
| makemv delim="," item_name
| makemv delim="," item_description
| eval mv_zipped=mvzip(item_name,item_description)
| mvexpand mv_zipped
| rex field=mv_zipped "(?P<ITEM_NAME>.*),(?P<ITEM_DESP>.*)"
| makemv delim="_" ITEM_DESP
| table _time ITEM_NAME ITEM_DESP

Although the purpose can be fulfilled by the following code. 

| mvexpand ITEM_DESP
| eval ITEM_DESP_char=printf("%c",ITEM_DESP)
| eventstats list(ITEM_DESP_char) as ITEM_DESP_char by ITEM_NAME
| eval ITEM_DESP_join=mvjoin(ITEM_DESP_char,"")
| dedup ITEM_NAME _time 
| table _time ITEM_NAME ITEM_DESP_join

Output:

_time	ITEM_NAME	ITEM_DESP_join
XXX	Name_1	ABC
YYY	Name_2	DEF
ZZZ	Name_3	GHI
000	Name_4	JKL
111	Name_5	MNO

If the item_description becomes very long(ex. lengh=50) and lots of items (ex. 50 items), the mvexpand command can't work properly with the output message below.
Error message:
command.mvexpand: output will be truncated at 28200 results due to excessive memory usage. Memory threshold of 500MB as configured in limits.conf / [mvexpand] / max_mem_usage_mb has been reached.

Is there any other way to transfer decimal value into ASCII and make the output as a string without using mvexpand command?

Thank you very much.

[yuanliu]

Do you mean mvmap?

| makeresults
| eval item_name = "Name_1,Name_2,Name_3,Name_4,Name_5", item_description = "65_66_67,68_69_70,71_72_73,74_75_76,77_78_79"
| makemv delim="," item_name
| makemv delim="," item_description
| eval mv_zipped=mvzip(item_name,item_description)
| mvexpand mv_zipped
| rex field=mv_zipped "(?P<ITEM_NAME>.*),(?P<ITEM_DESP>.*)"
| makemv delim="_" ITEM_DESP
| table _time ITEM_NAME ITEM_DESP
| eval ITEM_DESP_char=mvmap(ITEM_DESP, printf("%c",ITEM_DESP))

ITEM_NAME	ITEM_DESP	ITEM_DESP_char
Name_1	65 66 67	A B C
Name_2	68 69 70	D E F
Name_3	71 72 73	G H I
Name_4	74 75 76	J K L
Name_5	77 78 79	M N O

[Jouman]

Hi Yuan,

Thank you for the reply.

The key issue in my code is, if the items or the item description become very long, there will be warning messages for mvexpand due to too many memory usage.

Is there any other method to avoid using mvexpand while converting decimal values into text array?

Thank you.

[yuanliu · Accepted solution]

Have you tried my mvmap example?  It operates "locally" so to speak, processing one field in one event at a time.  mvexpand creates a ton of extra events (rows) that carry everything from the original row, therefore demands a lot more memory.

[Jouman]

Hi @yuanliu ,

Sorry for misunderstanding your reply at first. I saw mvexpand and I thought mvexpand() is part of the solution.

Thank you to introduce mvmap(). 
This command solves the issue indeed.

Thank you very much!