Splunk Search

How to extract fields from a field?

Motivator

Hi,

I have a field defined as message_text and it has entries like the below. It also has other entries that differ substantially from the example below. I'd like to extract the Remote IP Address, Session Id, and the credentials into other fields. Is this possible?

Relevant example:

message_text
    fn.webapp.listener.AuthenticationListener - Authentication event AuthenticationFailureBadCredentialsEvent: shahzadrazi; details: RemoteIpAddress: 111.88.102.145; SessionId: null; exception: Bad credential; nested exception is org.springframework.security.BadCredentialsException: Bad credential

Non-relevant examples:

message_text
    fn.ichealthcheck.servlet.ServerStatusServlet - Exiting doGet of ServerStatusServlet: 5 milliseconds

message_text
    org.jivesoftware.util.Log - SENT: <iq uri="/hb" type="result" id="1469534584" to="1003952@xmpp/0876ff2947cb"/
1 Solution

Legend

Try like this

*UPDATED*

.... | rex field=message_text "AuthenticationFailureBadCredentialsEvent:\s(?<credentials>[^\;]+)" | rex field=message_text "RemoteIpAddress:\s(?<RemoteIPAddress>[^\;]+)" | rex field=message_text "SessionId:\s(?<SessionId>[^\;]+)" | table credentials RemoteIPAddress SessionId

View solution in original post

Esteemed Legend

The command that you need to do this ad-hoc is rex.

http://docs.splunk.com/Documentation/Splunk/6.4.2/SearchReference/Rex

You might use it like this

... | rex field=message_text "RemoteIpAddress:\s*(?<RemoteIpAddress>[^\s;]+)"
0 Karma

Legend

Try like this

*UPDATED*

.... | rex field=message_text "AuthenticationFailureBadCredentialsEvent:\s(?<credentials>[^\;]+)" | rex field=message_text "RemoteIpAddress:\s(?<RemoteIPAddress>[^\;]+)" | rex field=message_text "SessionId:\s(?<SessionId>[^\;]+)" | table credentials RemoteIPAddress SessionId

View solution in original post

Motivator

Your comment looks like it should work but I'm getting an odd error.....

Here is the query
index=wls sourcetype=wlsmanagedserver | rex field=messagetext "AuthenticationFailureBadCredentialsEvent:\s(?[^\;]+) | rex field=messagetext "RemoteIpAddress:\s(?[^\;]+) | rex field=messagetext "SessionId:\s(?[^\;]+) | table credentials RemoteIPAddress SessionId

The error I'm getting is
Error in 'SearchParser': Missing a search command before '^'. Error at position '217' of search query 'search index=wls sourcetype=wlsmanagedserver bea...{snipped} {errorcontext = PAddress>[^\;]+) | re}'.

0 Karma

Legend

I have a suspicion, you are missing a < before IPAddress in the rex command for IPAddress.

 .... | rex field=message_text "AuthenticationFailureBadCredentialsEvent:\s(?<credentials>[^\;]+) | rex field=message_text "RemoteIpAddress:\s(?<RemoteIPAddress>[^\;]+) | rex field=message_text "SessionId:\s(?<SessionId>[^\;]+) | table credentials RemoteIPAddress SessionId
0 Karma

Motivator

Hmmm ok, I feel like an idiot 🙂 I can't seem to figure out what you are trying to tell me (sorry for the hard-headed-ness). Here is my current query (well really yours 🙂 )

index=wls sourcetype=wls_managedserver | rex field=message_text "AuthenticationFailureBadCredentialsEvent:\s(?[^\;]+) | rex field=message_text "RemoteIpAddress:\s(?[^\;]+) | rex field=message_text "SessionId:\s(?[^\;]+) | table credentials RemoteIPAddress SessionId

Still getting that error in SearchParser message

0 Karma

Legend

I think I know what wrong. Try the updated version

0 Karma

Motivator

Hi Sundareshr,

You saved my bacon..... again! Many many thanks!!!

0 Karma

Motivator
index=wls sourcetype=wls_managedserver | rex field=message_text "AuthenticationFailureBadCredentialsEvent:\s(?< credentials >[^\;]+) | rex field=message_text "RemoteIpAddress:\s(?< RemoteIPAddress >[^\;]+) | rex field=message_text "SessionId:\s(?< SessionId >[^\;]+) | table credentials RemoteIPAddress SessionId
0 Karma

Motivator

I put spaces around each rex field so it would show up

0 Karma