topic Re: How to extract fields from a field? in Splunk Search

How to extract fields from a field?

dbcase — Wed, 27 Jul 2016 00:33:00 GMT

Hi,

I have a field defined as message_text and it has entries like the below. It also has other entries that differ substantially from the example below. I'd like to extract the Remote IP Address, Session Id, and the credentials into other fields. Is this possible?

Relevant example:

message_text
    fn.webapp.listener.AuthenticationListener - Authentication event AuthenticationFailureBadCredentialsEvent: shahzadrazi; details: RemoteIpAddress: 111.88.102.145; SessionId: null; exception: Bad credential; nested exception is org.springframework.security.BadCredentialsException: Bad credential

Non-relevant examples:

message_text
    fn.ichealthcheck.servlet.ServerStatusServlet - Exiting doGet of ServerStatusServlet: 5 milliseconds

message_text
    org.jivesoftware.util.Log - SENT: <iq uri="/hb" type="result" id="1469534584" to="1003952@xmpp/0876ff2947cb"/

Re: How to extract fields from a field?

sundareshr — Wed, 27 Jul 2016 00:43:37 GMT

Try like this

*UPDATED*

.... | rex field=message_text "AuthenticationFailureBadCredentialsEvent:\s(?<credentials>[^\;]+)" | rex field=message_text "RemoteIpAddress:\s(?<RemoteIPAddress>[^\;]+)" | rex field=message_text "SessionId:\s(?<SessionId>[^\;]+)" | table credentials RemoteIPAddress SessionId

Re: How to extract fields from a field?

woodcock — Wed, 27 Jul 2016 00:46:02 GMT

The command that you need to do this ad-hoc is rex.

http://docs.splunk.com/Documentation/Splunk/6.4.2/SearchReference/Rex

You might use it like this

... | rex field=message_text "RemoteIpAddress:\s*(?<RemoteIpAddress>[^\s;]+)"

Re: How to extract fields from a field?

dbcase — Tue, 29 Sep 2020 10:22:32 GMT

Your comment looks like it should work but I'm getting an odd error.....

Here is the query
index=wls sourcetype=wls_managedserver | rex field=message_text "AuthenticationFailureBadCredentialsEvent:\s(?[^\;]+) | rex field=message_text "RemoteIpAddress:\s(?[^\;]+) | rex field=message_text "SessionId:\s(?[^\;]+) | table credentials RemoteIPAddress SessionId

The error I'm getting is
Error in 'SearchParser': Missing a search command before '^'. Error at position '217' of search query 'search index=wls sourcetype=wls_managedserver bea_...{snipped} {errorcontext = PAddress>[^\;]+) | re}'.

Re: How to extract fields from a field?

sundareshr — Wed, 27 Jul 2016 02:16:32 GMT

I have a suspicion, you are missing a < before IPAddress in the rex command for IPAddress.

 .... | rex field=message_text "AuthenticationFailureBadCredentialsEvent:\s(?<credentials>[^\;]+) | rex field=message_text "RemoteIpAddress:\s(?<RemoteIPAddress>[^\;]+) | rex field=message_text "SessionId:\s(?<SessionId>[^\;]+) | table credentials RemoteIPAddress SessionId

Re: How to extract fields from a field?

dbcase — Wed, 27 Jul 2016 02:41:50 GMT

Hmmm ok, I feel like an idiot 🙂 I can't seem to figure out what you are trying to tell me (sorry for the hard-headed-ness). Here is my current query (well really yours 🙂 )

index=wls sourcetype=wls_managedserver | rex field=message_text "AuthenticationFailureBadCredentialsEvent:\s(?[^\;]+) | rex field=message_text "RemoteIpAddress:\s(?[^\;]+) | rex field=message_text "SessionId:\s(?[^\;]+) | table credentials RemoteIPAddress SessionId

Still getting that error in SearchParser message

Re: How to extract fields from a field?

dbcase — Wed, 27 Jul 2016 02:44:41 GMT

index=wls sourcetype=wls_managedserver | rex field=message_text "AuthenticationFailureBadCredentialsEvent:\s(?< credentials >[^\;]+) | rex field=message_text "RemoteIpAddress:\s(?< RemoteIPAddress >[^\;]+) | rex field=message_text "SessionId:\s(?< SessionId >[^\;]+) | table credentials RemoteIPAddress SessionId

Re: How to extract fields from a field?

sundareshr — Wed, 27 Jul 2016 02:45:04 GMT

I think I know what wrong. Try the updated version

Re: How to extract fields from a field?

dbcase — Wed, 27 Jul 2016 02:45:12 GMT

I put spaces around each rex field so it would show up

Re: How to extract fields from a field?

dbcase — Wed, 27 Jul 2016 15:51:31 GMT

Hi Sundareshr,

You saved my bacon..... again! Many many thanks!!!