Getting Data In

Fields search doesn't work when filtering internal fields

sansay
Contributor

Here is the search string:

MissingUserData exchange rate | rex "ID :(?.+)" | fields ORDERID

This returns 8 records on a specific time frame. But it returns the whole log string.

If I add " | fields - _*"

to only get the values of ORDERID in the results like this:

MissingUserData exchange rate | rex "ID :(?.+)" | fields ORDERID | fields - _*

I get nothing.

That behavior just baffles me. Anyone has an idea why that might be?

Tags (1)
0 Karma

Runals
Motivator

I'm just a noob but I think the results of the search will be "ORDERID=2646042148." If you need to pass just "2646042148" you might need to rename the ORDERID field to query and return just that. I think I read this on post Ayn responded to once. Your query (subquery actually) would be something like:

[ MissingUserData exchange rate | rex "ID :(?<ORDERID>.+)" | rename ORDERID as query | fields query ]

I did that for a query where I was looking up IPs in a lookup table and throwing them against some very unstructured data. My query was as follows where the column name in test2.csv was IP. This does impact search performance and like I said I'm a noob so in the interest of full disclosure YMMV

index=netflow [| inputlookup test2.csv | rename IP as query | fields query ]

Note the following link: http://docs.splunk.com/Documentation/Splunk/4.3.4/User/HowSubsearchesWork

sansay
Contributor

Thank you Runals. You hit it on the nail with your answer.

0 Karma

Ayn
Legend

"The whole log string" is stored in the field _raw, so when you do | fields - _* you're removing that field as well, which is why you're not seeing any events - you should only be seeing your values for ORDERID.

0 Karma

sansay
Contributor

Here is a sample log that shows up, instead of just giving me the values for ORDERID:
2012-02-06T03:07:54.739-08:00 lvp-p2-nppaysys01 BibitServer[9081]: WARNING: LogCategory [MissingUserData] No exchange rate for Order ID :2646042148

Note that I can get only ORDERID values displayed with:

MissingUserData exchange rate | rex "ID :(?<ORDERID>.+)" | fields ORDERID | table ORDERID

But then I can't use that to feed it to another search, as it fails.

0 Karma
Get Updates on the Splunk Community!

Splunk Observability Cloud | Unified Identity - Now Available for Existing Splunk ...

Raise your hand if you’ve already forgotten your username or password when logging into an account. (We can’t ...

Index This | How many sides does a circle have?

February 2024 Edition Hayyy Splunk Education Enthusiasts and the Eternally Curious!  We’re back with another ...

Registration for Splunk University is Now Open!

Are you ready for an adventure in learning?   Brace yourselves because Splunk University is back, and it's ...