topic Re: How to create a regex for a log file which contains multiple values throughout the log which required same field nam in Splunk Search

How to create a regex for a log file which contains multiple values throughout the log which required same field name?

mssoni — Wed, 14 Dec 2022 15:17:53 GMT

Hello Team,

This is the first time I am posting a question and hope that I have explained it thoroughly.

I am trying to create a regex for a log file which contains multiple values throughout the log which required same field name. but splunk does not allows to use same field name again.

Here is the sample log:

/TXT1/TXT2/NMBR1/NMBR2/NMBR3/NMBR4/NMBR5/NMBR6/NMBR7/NMBR8/NMBR9/TXT1/TXT2/NMBR1/NMBR2/NMBR3/NMBR4/NMBR5/NMBR6/NMBR7/NMBR8/NMBR9/TXT1/TXT2/NMBR1/NMBR2/NMBR3/NMBR4/NMBR5/NMBR6/NMBR7/NMBR8/NMBR9/TXT1/TXT2/NMBR1/NMBR2/NMBR3/NMBR4/NMBR5/NMBR6/NMBR7/NMBR8/NMBR9/TXT1/TXT2/NMBR1/NMBR2/NMBR3/NMBR4/NMBR5/NMBR6/NMBR7/NMBR8/NMBR9/TXT1/TXT2/NMBR1/NMBR2/NMBR3/NMBR4/NMBR5/NMBR6/NMBR7/NMBR8/NMBR9/TXT1/TXT2/NMBR1/NMBR2/NMBR3/NMBR4/NMBR5/NMBR6/NMBR7/NMBR8/NMBR9/TXT1/TXT2/NMBR1/NMBR2/NMBR3/NMBR4/NMBR5/NMBR6/NMBR7/NMBR8/NMBR9/TXT1/TXT2/NMBR1/NMBR2/NMBR3/NMBR4/NMBR5/NMBR6/NMBR7/NMBR8/NMBR9

Note: Text values are 4 char and Number contains 10 digits.

How can I move forward to achieve a field extraction and format like this?
/TXT1/TXT2/NMBR1/NMBR2/NMBR3/NMBR4/NMBR5/NMBR6/NMBR7/NMBR8/NMBR9
/TXT1/TXT2/NMBR1/NMBR2/NMBR3/NMBR4/NMBR5/NMBR6/NMBR7/NMBR8/NMBR9
/TXT1/TXT2/NMBR1/NMBR2/NMBR3/NMBR4/NMBR5/NMBR6/NMBR7/NMBR8/NMBR9
/TXT1/TXT2/NMBR1/NMBR2/NMBR3/NMBR4/NMBR5/NMBR6/NMBR7/NMBR8/NMBR9
/TXT1/TXT2/NMBR1/NMBR2/NMBR3/NMBR4/NMBR5/NMBR6/NMBR7/NMBR8/NMBR9
/TXT1/TXT2/NMBR1/NMBR2/NMBR3/NMBR4/NMBR5/NMBR6/NMBR7/NMBR8/NMBR9
/TXT1/TXT2/NMBR1/NMBR2/NMBR3/NMBR4/NMBR5/NMBR6/NMBR7/NMBR8/NMBR9
/TXT1/TXT2/NMBR1/NMBR2/NMBR3/NMBR4/NMBR5/NMBR6/NMBR7/NMBR8/NMBR9
/TXT1/TXT2/NMBR1/NMBR2/NMBR3/NMBR4/NMBR5/NMBR6/NMBR7/NMBR8/NMBR9

Thank You in Advance 😊

Re: How to create a regex for a log file which contains multiple values throughout the log which required same field nam

yuanliu — Wed, 14 Dec 2022 18:44:31 GMT

I am not sure what you mean by "Splunk does not allow to use the same field name again." (This is SPL, of course anything is possible®😃)

Because your field values are separated by a known, fixed string "/TXT1/TXT2", a literal solution would be

| eval samefield = split(_raw, "/TXT1/TXT2") | eval samefield = mvfilter(len(samefield) != 0) | eval samefield = mvmap(samefield, "/TXT1/TXT2" . samefield)

This is an emulation I used to test the above; you can play with it and compare with real data

| makeresults | eval _raw = "/TXT1/TXT2/NMBR1/NMBR2/NMBR3/NMBR4/NMBR5/NMBR6/NMBR7/NMBR8/NMBR9/TXT1/TXT2/NMBR1/NMBR2/NMBR3/NMBR4/NMBR5/NMBR6/NMBR7/NMBR8/NMBR9/TXT1/TXT2/NMBR1/NMBR2/NMBR3/NMBR4/NMBR5/NMBR6/NMBR7/NMBR8/NMBR9/TXT1/TXT2/NMBR1/NMBR2/NMBR3/NMBR4/NMBR5/NMBR6/NMBR7/NMBR8/NMBR9/TXT1/TXT2/NMBR1/NMBR2/NMBR3/NMBR4/NMBR5/NMBR6/NMBR7/NMBR8/NMBR9/TXT1/TXT2/NMBR1/NMBR2/NMBR3/NMBR4/NMBR5/NMBR6/NMBR7/NMBR8/NMBR9/TXT1/TXT2/NMBR1/NMBR2/NMBR3/NMBR4/NMBR5/NMBR6/NMBR7/NMBR8/NMBR9/TXT1/TXT2/NMBR1/NMBR2/NMBR3/NMBR4/NMBR5/NMBR6/NMBR7/NMBR8/NMBR9/TXT1/TXT2/NMBR1/NMBR2/NMBR3/NMBR4/NMBR5/NMBR6/NMBR7/NMBR8/NMBR9" ``` data emulation above ```

There can be several variants based on fixed text values.

If you want to relax the condition that the text portion is known and fixed, you can use the text and number characteristics you described,

| rex max_match=0 "(?<samefield>(/\w{4}){2}(/d{3}){9})"

You can even generalize this to only require a total of 11 path segments:

| rex max_match=0 "(?<samefield>(/[^\/]+){11})"

Re: How to create a regex for a log file which contains multiple values throughout the log which required same field nam

mssoni — Thu, 15 Dec 2022 03:49:00 GMT

the provided sample is a single log and is not separated by fields, but I want to implement fields with the help of regex in this format.

Re: How to create a regex for a log file which contains multiple values throughout the log which required same field nam

yuanliu — Thu, 15 Dec 2022 05:40:16 GMT

@mssoni Have you tried my code? Yes, I do understand that's one string and that's what I emulated: One very long string.

Re: How to create a regex for a log file which contains multiple values throughout the log which required same field nam

mssoni — Thu, 15 Dec 2022 23:58:03 GMT

Not yet, I am really confused how to place this in the query, this is the first time I am working on its front end,

Re: How to create a regex for a log file which contains multiple values throughout the log which required same field nam

yuanliu — Fri, 16 Dec 2022 06:18:52 GMT

really confused how to place this in the query

One way to help others help you unconfuse you is to illustrate the SPL you have at hand (simplify and anonymize as needed), then explain what confuses you, what error you get, or why the result is not what you wanted.

I already explained how I test my sample codes based on your sample data and description of your requirements. I assume that you know where the search window is. Just paste data emulation code above the manipulation code, and you can review the results. Like this:

(In this example, I used the shorted code.) Does this look like what you need? Then, just replace the data emulation part with your own event search.

Hope this helps.