Getting Data In
Highlighted

How to link fields with different names across sources?

Splunk Employee
Splunk Employee

I have two types of transactions, one coming from a mobile app when a push notification is sent, looks approx like this:

TIMESTAMP="2016-03-29 23:39:01" DDSDKAppEventPushNotificationDelivered Msg=536 UID=101053 DEVICE=galaxys6 IP=XXX

and events from a shopping cart system, where that user has placed an order, that look approx like this:

04/04/2016:15:26:15 TRANSPAYMENT SYSTEM=DD LABEL=1 MERCHANTID=XXX REGISTER=1 LOGIN=125451 TICKET=XXX PAID=YES PAYMETHOD=GIFTCARD CUSTLOYALTY=101053 ITEMS=1 SALE=2.73 DISCTYPE=FLAT DISCAMT=0.0 DISCAPPLIED=0

The CUSTLOYALTY field in the shopping cart is the same data as the UID field from the push notifications (if this was a SQL database, I would join tables on those fields). I have created a field alias for those two fields, "CUSTLOYALTY AS UID".

I've got some simple logic to separate the customers that received a push notification, coming from the mobile app log:

eval PromoRcvd = if (Msg>1,"Yes","No")

But when I try to mix fields from the mobile app log and the shopping cart log, the shopping cart fields (in this case, SALE) are always empty:

table UID,Msg,PromoRcvd,sum(SALE)

I thought I didn't need to explicitly join sources in Splunk in order to search across them?

Highlighted

Re: How to link fields with different names across sources?

SplunkTrust
SplunkTrust

For the short version, check out this excellent post by @MuS at http://answers.splunk.com/answers/129424/how-to-compare-fields-over-multiple-sourcetypes-without-joi...

For the long version, check out this excellent talk by @sideview at http://wiki.splunk.com/Virtual_.conf (March 2016).

In your example case, you'd do something like this:

search for datasource1 OR datasource2 | stats values(Msg) as Msg values(PromoRcvd) as PromoRcvd sum(SALE) by UID

The variations are endless.

Highlighted

Re: How to link fields with different names across sources?

Splunk Employee
Splunk Employee

Ok, that does some weird things in the results table but it does look like it linked the fields correctly. But I think it's also creating duplicates in the results?

https://www.dropbox.com/s/l4oghy8kdt3bsch/Screenshot%202016-04-04%2016.43.39.png?dl=0

0 Karma
Highlighted

Re: How to link fields with different names across sources?

SplunkTrust
SplunkTrust

Each UID is listed once, looks fine to me.

If you're referring to the Yes-No-mv-field, I guess there are events for those UIDs with both Yes and No values. What that means depends on your data and requirements.

0 Karma
Highlighted

Re: How to link fields with different names across sources?

SplunkTrust
SplunkTrust

Hi bnash [Splunk],

use stats instead of table:

 your base search to get all the events 
 | eval PromoRcvd = if (Msg>1,"Yes","No")
 | stats sum(SALE) AS SALE by UID, Msg, PromoRcvd

Hope this helps ....

cheers, MuS

Highlighted

Re: How to link fields with different names across sources?

Splunk Employee
Splunk Employee

That still leaves the SALE field blank:

index=pos OR index=mobileapp | eval PromoRcvd = if (Msg>1,"Yes","No") | stats sum(SALE) AS SALE by UID, Msg, PromoRcvd

https://www.dropbox.com/s/lxmkqtys6lf7fkp/Screenshot%202016-04-04%2016.42.33.png?dl=0

Did I do something wrong?

0 Karma
Highlighted

Re: How to link fields with different names across sources?

SplunkTrust
SplunkTrust

Can you provide some examples for @martin_mueller and I - please ?

0 Karma
Highlighted

Re: How to link fields with different names across sources?

Splunk Employee
Splunk Employee

Sure thing - examples of what? What data do you need?

0 Karma