regex in transform.conf to extract hostname after equal to sign

[dbashyam]

Hi, I need help in extracting the hostname after equal to sign in the transform.conf file. The string pattern is like this

cs1=host-name-01-02

I tried the pattern but it seems to not work.
REGEX = .+cs1=(\S+)

could someone help?

Thanks,
Dinesh

[joebisesi]

Have you tried using the Extract New Fields? If you go into "Extract New Fields" select the event that you want, select Regular Expression, then next. Highlight the text that you want extracted, give it a test name. You can then select 'Show Regular Expression', and it will give you a pretty good idea of what Splunk is looking for as far as Regular Expressions go. I have found this method to work most of the time. Using sites like regex101 is useful, although I have put Regex's that I pulled from the above method, not return the same results in regex101. You might have to tweak the regex that you get from the above method, but usually the tweaking is minimal.

Hope this helps.

[FrankVl]

Not sure if I would really agree with that suggestion. Yes it can give some inspiration, but the regexes generated that way are often overly complex and not the most sensible solution.

It is worth investing some time in learning regex and writing your own, by understanding the data structure, rather than relying on automated tools to generate stuff like this for you.

For example in this case, this is what Splunk comes up with: ^(?:[^=\n]*=){2}(?P<host>[^ ]+), while cs1=(\S+) should do just fine.

[joebisesi]

I agree that the regexes generated that way are overly complex. Although sometimes, at least what I have found, is the ones that are generated will work in Splunk, and the regexes that are you can create and test in other environments don't work. In addition, by generating the overly complex ones, you can put them in something like regex101 and you can start to understand regex and how to work with the data.

One question for you. Do you have or know of a good site to learn regex? I have not found any that I would recommend. I have learned by taking generated regexes and breaking them down. So I'm sure I have several regexes that are not written in the best way.

[FrankVl]

I also mostly learned by doing and trying to understand regexes created by others / found in splunkbase apps etc. So no, I don't have a site that is good for learning. Although regex101 does also have content (bottom right corner) explaining the various regex concepts, but that is a bit limited in the amount of explanation and examples given.

[joebisesi]

That's pretty much what I do

[FrankVl]

What does the rest of your props and transforms look like for this (maybe the issue isn't with the regex itself).

For the regex: that .+ is not very useful, try without it.

[dbashyam]

Hi,

the props.conf contains

[dbnetworks:monitor:file]
TRANSFORMS-DBN = DBNsyslog

the transforms.conf contains

[DBNsyslog]
REGEX = .+cs1=(\S+)
FORMAT = host::$1
DEST_KEY = MetaData:Host

[FrankVl]

I don't see anything obviously wrong with that. Perhaps try without the .+.

Also:
did you restart the respective splunk instance after adding this config?
You realize this will only apply to newly ingested events?
What does your splunk architecture look like for this data feed and where did you deploy this config in that architecture?

[dbashyam]

thanks @FrankVl your suggestion of removing the .+ did the trick.

[FrankVl]

That's good to hear! I converted my comment to an answer, so you can mark it as accepted.

[kamlesh_vaghela]

@dbashyam

I think cs1 should extract automatically. Can you please share full sample event ? So we can work on right path.

[dbashyam]

Hi, below is the sample.

Nov 28 10:11:06 fwrw dbn: arf:0|data|data|0.0.1|1|repeat_event|7|cs1Label=system identifier cs1=MM63-GH-CG-A-0101 system_identifier=DB63-GH-CG-A-0101 externalId=10043284 cnt=63784 rt=12345 start=123456 destinationServiceName=dummy cn1Label=statement identifier