Running rex within an eval/if

[genesiusj]

Hello,
I Googled and searched the Answers forum, but with no luck.
Below, in psuedo code, is what I want to accomplish.

eval newfield if oldfield starts with a double quote, newfield equals oldfield; if not, run a rex on oldfield.

I have the code for the rex from hex to text.

| rex mode=sed field=cmd "s/([0-9A-Fa-f]{2})/%\1/g" | eval cmd=urldecode(substr(cmd,1))

What I am looking for is how to search for a field containing double quotes using an if statement, and run the rex command if the field doesn't contain double quotes.

Example of two different values for old field and the result I need on newfield.

oldfield "dmidecode"
newfield "dmidecode" or dmidecode

oldfield 7375202D
newfield su -

AMENDED QUESTION
Sample events.

type=USER_CMD msg=audit(1570115120.149:13760551): user pid=29573 uid=1014 auid=1014 ses=27921 msg='cwd="" cmd="dmidecode" terminal=pts/0 res=success'

type=USER_CMD msg=audit(1570201689.897:13954761): user pid=702 uid=1014 auid=1014 ses=28314 msg='cwd="" cmd=7375202D terminal=pts/0 res=success'

In the first event, double quotes are part of the field value for cmd in _raw.
In the second event, there are no double quotes in the field value of cmd in _raw.

Thanks and God bless.
Genesius

[DavidHourani]

Hi @genesiusj,

Here's what I understand:
1- A field called old-value exists and you want to make a new field based on that.
2- IF oldfield has quotes THEN newfield equals oldfield.
3- IF oldfielddoesn't have quotes THEN newfield equals decode oldfield.

Supposing in your case old field is cmd, your search should look like this :

yourbasesearch | fields cmd | eval newfield=if(match(cmd,"^\".+"),urldecode(substr(cmd,1)), cmd) 

The logic sounds complicated but the query is pretty easy.

Let me know if that helps.

Cheers,
David

[pjpscriv]

I had a similar issue and this helped me - thanks! 🙏

[genesiusj]

@DavidHourani
Replaced oldfield with cmd; and newfield with cmdConverted.

| eval cmdConverted=if(match(cmd,^\".+),urldecode(substr(cmd,1)), cmd) 

Didn't work.
Error in 'eval' command: The expression is malformed. An unexpected character is reached at '^\".+),urldecode(substr(cmd,1)), cmd)'.

Thanks and God bless,
Genesius

[DavidHourani]

woops, missing quotes lol, fixed in answer and second comment ^^

[genesiusj]

@DavidHourani
Thank you.
Here is what I ran, replacing the field names as mentioned above.

| eval cmdConverted=if(match(cmd,"^\".+"),cmd, null()) 
| rex mode=sed field=cmd "s/([0-9A-Fa-f]{2})/%\1/g" 
| eval cmdConverted=if(isnull(cmdConverted),cmd,cmdConverted)

Here is what both fields now look like.

dmi%deco%de     dmi%deco%de

%73%75%20%2D     %73%75%20%2D

The code did not convert and it added % in between each set of hex digits, and before the 2nd and 3rd d in the text.
Strange.

Thanks and God bless,
Genesius

[DavidHourani]

Hi @genesiusj,

The sed should be applied to the cmdConverted field. Your search should be like this :

 | eval cmdConverted=if(match(cmd,"^\".+"),cmd, null()) 
 | rex mode=sed field=cmdConverted "s/([0-9A-Fa-f]{2})/%\1/g" 
 | eval cmdConverted=if(isnull(cmdConverted),cmd,cmdConverted)

Let me know that works out for you.

Cheers,
David

[ololdach]

Hi David, won't work, because in the case of urldecode, the oldfield needs to be rexed from ADDDEF to %AD%DD%EF ... just as an example.

[DavidHourani]

In that case you have to do 3- before 2- in the order of the logic above, your search should be like this :

 yourbasesearch | fields cmd | eval newfield=if(match(cmd,"^\".+"),cmd, null()) | LOGIC FOR FORMATING newfield AS REQUIRED THEN DECODING| eval newfield=if(isnull(newfield),cmd,newfield)

[ololdach · Accepted solution]

Please try:

| makeresults | eval raw="type=USER_CMD msg=audit(1570115120.149:13760551): user pid=29573 uid=1014 auid=1014 ses=27921 msg='cwd=\"\" cmd=\"dmidecode\" terminal=pts/0 res=success'|type=USER_CMD msg=audit(1570201689.897:13954761): user pid=702 uid=1014 auid=1014 ses=28314 msg='cwd=\"\" cmd=7375202D terminal=pts/0 res=success'" | makemv delim="|" raw | mvexpand raw | eval _raw=raw | fields - raw 
| rex field=_raw "cmd=(?<cmd>[^\s]+)" | rex field=cmd "\"(?<cleardata>[^\"]*)\""
| rex mode=sed field=cmd "s/([0-9A-Fa-f]{2})/%\1/g" | eval hexdata=urldecode(cmd) 
| eval newfield=if(isnull(cleardata),hexdata,cleardata)

Like before, anything up to the first rex is to reproduce the data. From there on, the logic is straightforward: Do both conversions on the cmd field. Every conversion fails in one case... so use the other.
Best - Oliver

[ololdach]

Hi genesiusj, I should have been less modest. This actually is the solution. Just run the above query. I have left the temporary fields intentionally, so that you can see how it works. Oliver

[genesiusj]

@ololdach
If you could re-post as a new answer

Replaced oldfield with cmd; and newfield with cmdConverted.

 | rex field=_raw "cmd=(?<cmd>[^\s]+)" | rex field=cmd "\"(?<cleardata>[^\"]*)\""
 | rex mode=sed field=cmd "s/([0-9A-Fa-f]{2})/%\1/g" | eval hexdata=urldecode(cmd) 
 | eval cmdConverted=if(isnull(cleardata),hexdata,cleardata)")`

Then I can accept and upvote.

Thanks and God bless,
Genesius

[genesiusj]

@ololdach

Here is your code with my added comments. If you feel this correct, please post as a new answer so I can accept and upvote.

  `comment("If the cmd field contains quotes in _raw, then the quotes will be added to the field extraction.")`
| rex field=_raw "cmd=(?<cmd>[^\s]+)" 
  `comment("Move the character after the first doublequote; capture all the text until the second doublequote.")`
| rex field=cmd "\"(?<cleardata>[^\"]*)\"" 
  `comment("Add a % before each hexadecimal digit. If the value of cmd is not all hex, there will be % after every 'supposed' hex digit.")`
| rex mode=sed field=cmd "s/([0-9A-Fa-f]{2})/%\1/g" 
  `comment("Convert cmd from hex to text. If the value of cmd is not all hex, there will no conversion.")`
| eval hexdata=urldecode(cmd) 
  `comment("If cleardata, which is the field representing the true text, is blank, use the hex conversion (hexdata). If not, use the cleardata (text) as the value of cmdConverted")`
| eval cmdConverted=if(isnull(cleardata),hexdata,cleardata)

Thanks again for all your help.

God bless,
Genesius

[genesiusj]

@ololdach
Replaced oldfield with cmd; and newfield with cmdConverted.

| rex field=_raw "cmd=(?<cmd>[^\s]+)" | rex field=cmd "\"(?<cleardata>[^\"]*)\""
| rex mode=sed field=cmd "s/([0-9A-Fa-f]{2})/%\1/g" | eval hexdata=urldecode(cmd) 
| eval cmdConverted=if(isnull(cleardata),hexdata,cleardata)")`

At first I didn't know hexdata and cleardata were other new fields. So I had replaced them with cmdConverted and cmd. 😞
But once I "learned" they were other new fields, this worked.

Thanks and God bless,
Genesius

[ololdach]

Hi Genesiusj, glad I could help. As a tip: If somebody gives you a query and you have no idea what he/she does, paste it step by step, one | at a time and take a look at how the results/logic work and develop. I do that myself whenever I feel rather stupid... which happens on a daily basis 🙂

[ololdach]

If I understand correctly, you would like to extract a new field newfield from some oldfield using rex, but only, if the oldfield has double quotes? I assume that the value of newfield should be the characters from inbetween the quotes. Now, if my assumptions are correct, I'd just use the rex: ...| rex field=oldfield "\"(?<newfield>[^\"]*)\""

Try the following query to reproduce:

| makeresults | eval oldfield="\"foo\",bar" | makemv delim="," oldfield | mvexpand oldfield 
| rex field=oldfield "\"(?<newfield>[^\"]*)\"" | ... do a rex to change the hex data into a new field named hexdata ... 
| eval newfield=if(isnull(newfield),hexdata,newfield)

Everything up to the rex is just to reproduce your sample data. After the rex to get the newfield, add the rex to convert the hex data, even if it produces only junk when newfield is set. All events where newfield is null, use the hexdata instead.

[genesiusj]

@ololdach
Close. Sort of.
Extract a new field newfield from some oldfield using rex, but only, if the oldfield has double quotes. BUT, if the oldfield doesn't contain double quotes we KNOW the value is in hex and not text. Therefore, a conversion from hex to text needs to be made on the oldfield and stored in the newfield.

Thanks and God bless,
Genesius

[ololdach]

See the ammendment above.

[genesiusj]

@ololdach
Thanks. I'm going to try that. While I do, I amended my question with two sample events. Apologies, I should have done this to begin with.
Thanks and God bless,
Genesius

[ololdach]

You were too fast 🙂 I didn't have time to type the ammendment and test it. Take a look. It should work