Developing for Splunk Cloud Services

can I combine values from multiple field extractions into a single table field

cbwillh
Path Finder

We have syslog data from our esxi hosts and typical of esxi logs there are multiple logs with different formats in the message portion of the log.
in short splunk cannot automatically determine which portion of the data is the message since all the logs have a string with nothing to identify the actual message we want to see.
we can see the messages if we do a search and just use verbose and look at the raw logs that are displayed. however I am trying to make it easier to read quickly by creating a table with _time, host, sourcetype, source message
note that "message" is where I want to see the message from all the different logs.

I have manually created field extractions but to get each logs message piece I had to create 9 separately named extractions which I have named:
message_hostd
message_hostdprobe
message_injector
message_rhttpproxy
message_sfcb
message_Vpxa
message_syslog
message_backup
message_crond

what I have been trying to do is funnel all of those field extraction names to the single "message" field in the table portion of my search.

I have been trying to use all of the currently available suggestions found on this site when I search for "combine multiple field extractions into a single table field" but nothing works.

sadly I am not very good with all of the splunk search commands beyond the basics and would really appreciate it if someone with much more knowledge can either point me in the right direction or confirm that what I am trying to do can not be done.

thank you all in advance for your help.

please note that I already have searched this splunk blog and I have tried everything even remotely similar to my described issue that already exists on this site with NO success. so I have definitely done my due diligence before posting here.

Tags (1)
1 Solution

woodcock
Esteemed Legend

You need the coalesce command like this:

... | eval message = coalesce(message_hostd, message_hostdprobe, message_injector, message_rhttpproxy, message_sfcb, message_Vpxa, message_syslog, message_backup, message_crond)

Also, you can do your field extractions in pairs like this:

EXTRACT-double = (?<message>(?<message_hostd>YourRegExHere))

View solution in original post

woodcock
Esteemed Legend

You need the coalesce command like this:

... | eval message = coalesce(message_hostd, message_hostdprobe, message_injector, message_rhttpproxy, message_sfcb, message_Vpxa, message_syslog, message_backup, message_crond)

Also, you can do your field extractions in pairs like this:

EXTRACT-double = (?<message>(?<message_hostd>YourRegExHere))

cbwillh
Path Finder

Hi Woodcock,
awesome that worked a treat! thanks so much for your help clarifying the command syntax.

I just tried it in my search and it worked exactly as I wanted!

I am awarding points to chrisyoungerjds as well for this. you guys both really helped me out here. very much appreciated!

0 Karma

chrisyounger
SplunkTrust
SplunkTrust

glad you solved it 🙂

0 Karma

chrisyounger
SplunkTrust
SplunkTrust

Yes good one. For some reason I thought coalesce didn't support more than two arguments, but there you go! Thanks.

0 Karma

chrisyounger
SplunkTrust
SplunkTrust

This is one way to do it:

| eval message = coalesce(coalesce(coalesce(coalesce(coalesce(coalesce(coalesce(coalesce(message_hostd, message_hostdprobe), message_injector) , message_rhttpproxy) , message_sfcb) , message_Vpxa) , message_syslog) , message_backup ), message_crond)

Oh, I should also mention that if you want, you can have multiple field extractions defined that set the same field value. The first one (in ascii order) will take effect if there are multiple matches. This way you don't need to do the coalescing in your search query.

0 Karma

cbwillh
Path Finder

Hi Chrisyoungerjds,
thanks so much for your help, I chose the answer from woodcock because he clarified the correct syntax but I wish I could put you both as your suggestion (once I used woodcocks syntax) did work perfectly. I really appreciate your help and I gave you points in lieu of choosing as the answer. thanks again your answer got me on the right track and with the syntax provided by woodcock it worked and did exactly what I was looking for.

0 Karma