Splunk Search

How could I correlate values in multiple fields that contain multiple values

digital_alchemy
Path Finder

So, I've been away from Splunk for several years now, and now re-visiting it.  I've got a scenario where I would like to track certain metrics from imported data.  I created a simple CSV with just a few entries to demonstrate the issues I'm having.

Below is the source data I created:

customer_idTimecustomer_fnamecustomer_lnameproductsproduct_prices
11112/1/2023JohnDoeproduct_100,product_200100,200
22212/11/2023SuzyQueproduct_100100
33312/15/2023JackJonesproduct_300300
11112/18/2023JohnDoeproduct_400400

 

In this scenario this is just examples of customers and the items they purchased and the price paid.

 

After uploading the file and displaying the data in a table it looks as expected:

source="test_sales.csv"
| table customer_id,customer_fname,customer_lname,products,product_prices

Basic_Table_Search.png

Upon using makemv to convert "products" and "product_prices" to multi-value fields, again the results are as expected and the product and price align since they were input into the source CSV in the proper order:

source="test_sales.csv"
| makemv delim="," products
| makemv delim="," product_prices
| table customer_id,customer_fname,customer_lname,products,product_prices

makemv_table.png

 

Here is where my issue is,  Is there a way to tie the product for a purchase transaction  in the multi-value "products" column to it's corresponding price in the multi-value "product_prices" column?

Everything seems to work except when I try to so something like listing the products by price for the multi-value fields like this:

source="test_sales.csv"
| makemv delim="," products
| makemv delim="," product_prices
| stats count(products) by products,product_prices

product_by_price.png

In the above results you can see that I'm getting results that are not exactly what I would want. 

Ex.  it shows:

3 instances of product_100 at a price of 100, should only be 2 instances

2 instances of product_100 at a price of 200, should be 0 instances of this combination

2 instances of product_200 at a price of 100, should be 0 instances of this combination

2 instances of product_200 at a price of 200, should only be 1 instance

 

I'm likely approaching this incorrectly or using the wrong tool for the task,  any help to get me on the right track would be appreciated.

 

Thanks

Labels (3)
0 Karma

PickleRick
SplunkTrust
SplunkTrust

Multivalued fields are separate entities which means Splunk doesn't keep any "connection" between values in those fields. For Splunk each field is just a single "multivalued value" (yes, I know it sounds bad ;-)).

So you have to manually combine those values. One solution @ITWhisperer already showed but for me it's a bit "brute force". My idea of a more "splunky" approach to splitting those products and product_prices would be to do

| eval zipped=mvzip(products,product_prices,":")
| mvexpand zipped
| eval zipped=split(zipped,":")
| eval products=mvindex(zipped,0), product_prices=mvindex(zipped,1)

Then you can do your stats

0 Karma

ITWhisperer
SplunkTrust
SplunkTrust
| eval row=mvrange(0,mvcount(products))
| mvexpand row
| eval products=mvindex(products, row)
| eval product_prices=mvindex(product_prices, row)
| stats count(products) by products,product_prices
0 Karma

digital_alchemy
Path Finder

@PickleRick @ITWhisperer   Thanks for the responses,  I tried both and they both give me the same result, but still not exactly what I had in mind in my head.

Here's the result:

unsep.png

 

What I'm trying to do is more along the lines of being able to get a count of products that sold at specific price points from transactions that may have multiple items purchased.

Maybe what I'm trying to do isn't really possible or the best approach to the problem,  but the following table shows kinda what I'm trying to accomplish.

productsproduct_pricescount(products)
product_1001002
product_2002001
product_3003001
product_4004001
0 Karma

ITWhisperer
SplunkTrust
SplunkTrust

Here is the result when I did it

ITWhisperer_0-1705912775121.png

 

Please share your search for when you didn't get the result you were expecting

0 Karma

PickleRick
SplunkTrust
SplunkTrust

Yep. My version works too

PickleRick_0-1705913465000.png

 

(of course if you do stats afterwards and not include the "zipped" field, it will vanish).

0 Karma
Get Updates on the Splunk Community!

Notification Email Migration Announcement

The Notification Team is migrating our email service provider from Postmark to AWS Simple Email Service (SES) ...

Mastering Synthetic Browser Testing: Pro Tips to Keep Your Web App Running Smoothly

To start, if you're new to synthetic monitoring, I recommend exploring this synthetic monitoring overview. In ...

Splunk Edge Processor | Popular Use Cases to Get Started with Edge Processor

Splunk Edge Processor offers more efficient, flexible data transformation – helping you reduce noise, control ...