Can you pass a regex as field value to the rex command

[Keith_NZ]

Hi All,

In SPL2 Ingest Pipeline I want to assemble a regular expression and then use that in a rex command but I am having trouble.

For example this simple test I am specifying the regex as a text string on the rex command works:

But this version doesnt:

Any idea what I am doing wrong?

Thanks

[livehybrid]

Hi @Keith_NZ 

I dont have an Ingress Processor instance available at the moment to test, but would a custom function work for you here?

Something like this?

function my_rex($source, $field, $rexStr: string="(?<all>.*)") {
    return | rex field=$field $rexStr 
}

FROM main | my_rex host "(?<hostname>.mydomain.com"

Please let me know how you get on and consider adding karma to this or any other answer if it has helped.
Regards

Will

[Keith_NZ]

Thanks _ I will have a look into this.

[ITWhisperer]

There are a couple of ways to do this but it depends on the context. For example, are you creating a dashboard? Where does the regex come from? Is it static? What is your use case? The more information you can provide, the more likely we will be able to give you useful suggestions.

[Keith_NZ]

This is for an Ingest Processor pipeline.  I have hundreds of fields I want to redact as events pass through the pipeline .  The regex for each includes the fieldname to look for and lots of complex regex for the various formatting options.

So rather than repeat that complex formatting lots of times I was thinking of a loop to loop through a list of the field names, assemble the regex, then process it.

Or maybe just a command for each fielname to search for that calls a function that assembles the regex and executes the rex command.

But I am starting to thing that SPL2 cant do this.  I better go do some more ready.

thanks

[gcusello]

Hi @Keith_NZ ,

at first, please in addition to the screenshots, add also the code and a sample of your logs in text format using the "Add/Edit Code sample" button.

Then,

if you are doing an extraction from _raw you don't need to explicit it in field option.

At least, your first rex expressio is almost correct, you have to declare the format of the field (e.g. if it's numeric you have to add \d, something like this, then you have to declare something to define the string to extract as field, e.g. to extract the postCode, you should use:

rex "postCode\\\":\\\"(?<postCode>\d+)"

in this specific case beware when you have backslashes because to use in Splunk you have to use an additional backslash.

Instead isn't correct the last one:

| rex field=_raw reg_str

because it isn't a field extraction.

Ciao.

Giuseppe

[Keith_NZ]

Hi Giuseppe,

Thanks for showing an interest.

I will try to include everything with this example:

The code is:

$pipeline = | from $source 
| rex "postCode:(?P<postCode1>\\d+)"
| eval regexstrA= "postCode:(?P<postCodeA>\\d+)"
| eval regexstrB= "postCode:(?P<postCodeB>\\\\d+)"
//| rex  regexstrA
//| rex  regexstrB
| into $destination;

the sample data is 

blah blah postCode:4548 blah blah

when I run it you can see the field value extracts properly and the fields in lines 3 and 4 also get created and you can see their contents:

But if I run with line 5 uncommented I get this error:

Error in 'rex' command: The regex 'regexstrA' does not extract anything. It should specify at least one named group. Format: (?<name>...).

and a similar error if I uncomment line 6.

Any ideas why?

Thanks