Getting Data In

Fields search doesn't work when filtering internal fields

sansay
Contributor

Here is the search string:

MissingUserData exchange rate | rex "ID :(?.+)" | fields ORDERID

This returns 8 records on a specific time frame. But it returns the whole log string.

If I add " | fields - _*"

to only get the values of ORDERID in the results like this:

MissingUserData exchange rate | rex "ID :(?.+)" | fields ORDERID | fields - _*

I get nothing.

That behavior just baffles me. Anyone has an idea why that might be?

Tags (1)
0 Karma

Runals
Motivator

I'm just a noob but I think the results of the search will be "ORDERID=2646042148." If you need to pass just "2646042148" you might need to rename the ORDERID field to query and return just that. I think I read this on post Ayn responded to once. Your query (subquery actually) would be something like:

[ MissingUserData exchange rate | rex "ID :(?<ORDERID>.+)" | rename ORDERID as query | fields query ]

I did that for a query where I was looking up IPs in a lookup table and throwing them against some very unstructured data. My query was as follows where the column name in test2.csv was IP. This does impact search performance and like I said I'm a noob so in the interest of full disclosure YMMV

index=netflow [| inputlookup test2.csv | rename IP as query | fields query ]

Note the following link: http://docs.splunk.com/Documentation/Splunk/4.3.4/User/HowSubsearchesWork

sansay
Contributor

Thank you Runals. You hit it on the nail with your answer.

0 Karma

Ayn
Legend

"The whole log string" is stored in the field _raw, so when you do | fields - _* you're removing that field as well, which is why you're not seeing any events - you should only be seeing your values for ORDERID.

0 Karma

sansay
Contributor

Here is a sample log that shows up, instead of just giving me the values for ORDERID:
2012-02-06T03:07:54.739-08:00 lvp-p2-nppaysys01 BibitServer[9081]: WARNING: LogCategory [MissingUserData] No exchange rate for Order ID :2646042148

Note that I can get only ORDERID values displayed with:

MissingUserData exchange rate | rex "ID :(?<ORDERID>.+)" | fields ORDERID | table ORDERID

But then I can't use that to feed it to another search, as it fails.

0 Karma
Get Updates on the Splunk Community!

Splunkbase | Splunk Dashboard Examples App for SimpleXML End of Life

The Splunk Dashboard Examples App for SimpleXML will reach end of support on Dec 19, 2024, after which no new ...

Understanding Generative AI Techniques and Their Application in Cybersecurity

Watch On-Demand Artificial intelligence is the talk of the town nowadays, with industries of all kinds ...

.conf24 | Registration Open!

Hello, hello! I come bearing good news: Registration for .conf24 is now open!   conf is Splunk’s rad annual ...