topic Re: Extract not giving the exact result in Splunk Search

Extract not giving the exact result

sunnyparmar — Mon, 21 Sep 2015 07:09:08 GMT

Hi,

I have an extract with the name "remotesystemid" but when i am executing the below query it is giving values with null column.

index=abc sourcetype=xyz | timechart count by remotesystemid

If i am using ( | stats values(*) as * by remotesystemid ) with the above given command to exclude null value then it is giving no result found.

Kindly suggest where i am wrong?

Thanks

Re: Extract not giving the exact result

richgalloway — Mon, 21 Sep 2015 11:04:22 GMT

It would seem Splunk is not finding your remotesystemid field. What does your data look like? Have you tried index=abc sourcetype=xyz | table remotesystemid?

Re: Extract not giving the exact result

sunnyparmar — Mon, 21 Sep 2015 11:48:25 GMT

thanks for the reply.. yes i have tried the same but it is giving "no result found".. Do you have any idea what to do in this case? thanks

Re: Extract not giving the exact result

richgalloway — Mon, 21 Sep 2015 11:55:38 GMT

Post a sample of your data so we can help you extract the remotesystemid field.

Re: Extract not giving the exact result

sunnyparmar — Mon, 21 Sep 2015 12:32:14 GMT

Now i am getting something by below query but its give me values with "null" column and timeframe column where as time frame column is showing right time and null column showing the right values but i don't need values with null column name so for that I have used stats command with the below query (| stats values(*) as * by remotesysid) but then again it is giving me "no result found"..

index=abc sourcetype=xyz | timechart count by remotesysid

Re: Extract not giving the exact result

Richfez — Mon, 21 Sep 2015 12:33:34 GMT

I think it's going wrong somewhere on the sourcetype=xyz assignment. That's not getting done for some reason so nothing that depends on it happens either. If it were getting done, the search would return a bunch of blanks, not "no results found" (I think).

Did the host change IP addresses or something?

Re: Extract not giving the exact result

sunnyparmar — Mon, 21 Sep 2015 12:34:26 GMT

My data is like given below -

INFO [http-8080-Processor22] 09-15 15:22:40 RemoteSystemId is nullcom.basware.bt.access.RemoteSystemIDNullException: Remote-System is NULL! Check the URL (MessengerServlet.java:257)
INFO [http-8080-Processor24] 09-15 15:21:40 RemoteSystemId is nullcom.basware.bt.access.RemoteSystemIDNullException: Remote-System is NULL! Check the URL (MessengerServlet.java:257)

Re: Extract not giving the exact result

sunnyparmar — Mon, 21 Sep 2015 12:41:49 GMT

Everything is handling by my system. Logs are placed in my local system. sourcetype is defined in my local system inputs.conf file and i am pushing logs on the server by splunk forwarder from my local system.

Re: Extract not giving the exact result

richgalloway — Mon, 21 Sep 2015 12:42:53 GMT

What is the [xyz] stanza of your props.conf file?

Re: Extract not giving the exact result

sunnyparmar — Mon, 21 Sep 2015 12:48:00 GMT

sorry but didn't get you? could you please elaborate more?

Re: Extract not giving the exact result

DennisMohn — Tue, 29 Sep 2020 07:19:07 GMT

How is the field "remotesystemid" extracted? You should have an entry in your props.conf file, which is located either in $SPLUNK_HOME/etc/system/local/ or $SPLUNK_HOME/etc/users/yourusername/search/local/ - depending if the field extraction is public or private

Please refer to the link @richgalloway has posted in his comment for further info about the props.conf

Re: Extract not giving the exact result

richgalloway — Mon, 21 Sep 2015 13:00:12 GMT

Once the logs get to your local system (the indexer), there should be a props.conf file describing how the xyz sourcetype should be handled. The relevant portion of that file will begin with "[xyz]". Please share that text, if it exists. If it doesn't exist, then we've found your problem.

Re: Extract not giving the exact result

sunnyparmar — Mon, 21 Sep 2015 13:11:21 GMT

First on splunk server its found under user directory and it has following entries under it -

[sc-kofax-extracts]
[sc-nova-email]
[ng-pay]

With this it is found under (etc/system/local/) as well and entries are

[my-onp-front]
TRANSFORMS-drop_noise = heartbeat

and on my local system from where i am pushing the data to splunk server through universal forwarder, its found under ($SPLUNK_HOME\SplunkUniversalForwarder\etc\system\default) and under this there are no entries related to "remotesystemid"

Re: Extract not giving the exact result

sunnyparmar — Mon, 21 Sep 2015 13:11:44 GMT

First on splunk server its found under user directory and it has following entries under it -

[sc-kofax-extracts]
[sc-nova-email]
[ng-pay]

With this it is found under (etc/system/local/) as well and entries are

[my-onp-front]
TRANSFORMS-drop_noise = heartbeat

and on my local system from where i am pushing the data to splunk server through universal forwarder, its found under ($SPLUNK_HOMESplunkUniversalForwarderetcsystemdefault) and under this there are no entries related to "remotesystemid"

Re: Extract not giving the exact result

DennisMohn — Mon, 21 Sep 2015 13:14:16 GMT

So this seems to be the problem. The field "remotesystemid" is never extracted, so Splunk does not know how to handle your request.

I suggest you check out the Field Extractor Manual: http://docs.splunk.com/Documentation/Splunk/6.2.6/Knowledge/ExtractfieldsinteractivelywithIFX

After you have added a field extraction for remotesystemid, you can go on with your search.

Re: Extract not giving the exact result

richgalloway — Mon, 21 Sep 2015 13:24:57 GMT

I don't see an [xyz] stanza in your etc/system/local/props.conf file. That means Splunk has no instructions about how to process that sourcetype and won't know how to find the remotesystemid field.

Re: Extract not giving the exact result

sunnyparmar — Mon, 21 Sep 2015 13:52:35 GMT

Hey Dennis.. the page you have shared with me and the procedure that is mentioned on the page for making field extraction, i have made my field extraction in the same way as it is mentioned on the page so do you have any idea why Splunk behave weird. Thanks

Re: Extract not giving the exact result

sunnyparmar — Mon, 21 Sep 2015 13:54:27 GMT

My logs are like given below from which i am making extraction "remotesystemid"

Then why Splunk not taking it as a extraction? do you have any suggestions? Thanks

Re: Extract not giving the exact result

DennisMohn — Mon, 21 Sep 2015 13:55:27 GMT

If you made the field extraction like mentioned in the tutorial, there should be an entry in your local props.conf file for it.

Can you check "Settings => Fields => Field Extractions" if your extraction shows up? Please post the regex under "Extractions/Transform"

Re: Extract not giving the exact result

sunnyparmar — Mon, 21 Sep 2015 14:09:25 GMT

Hey Dennis,

the last comment that you have posted i am not able to see that comment on this forum but though getting alert on my mail id.. so on basis of your questionbelow given is the "Extractions/Transform"

(?i) Removed (?P[^ ]+)

Thanks