Getting Data In

Search the table values in multi-values csv files

pratheep1980
New Member

From the log, i've extracted the required columns (around 10 columns) and formed a table with values. I want to search the table values with csv file having multi-valued in few columns and retrieve the output of other column.

Please help to search in multi-valued fields and fetch the required field value.

Query used to create the table:
4091200 | eval casetime=strftime(_time, "%d-%m-%Y %H:%M:%S")|table PROCESS_KEY MEMBER_SOLD_STATE REVIEW_TYPE FUNDING_TYPE CASE_TYPE REQUEST_TYPE SRVC_LINE_STS SRVC_LINE_STS_RSN LEVEL_OF_SERVICE LENGTH_OF_STAY PREREQUISITES_TIME NOTIFICATION_TYPE ERISA_INDICATOR processOutcome agendaGroup casetime| where processOutcome like "%REQ%" or processOutcome like "%CURR%" | sort by casetime desc | dedup casetime MEMBER_SOLD_STATE REVIEW_TYPE FUNDING_TYPE REQUEST_TYPE SRVC_LINE_STS SRVC_LINE_STS_RSN LEVEL_OF_SERVICE LENGTH_OF_STAY PROCESS_KEY

alt text

Tags (2)
0 Karma

woodcock
Esteemed Legend

Show a few lines of the lookup file. Show a few sample events. Show a mockup of the desired output. Include test of logic that generates this output. As it stands now, nothing is clear enough for anybody to help you.

0 Karma

pratheep1980
New Member

@ woodcock: I am unable to attach the samples here.. hence sent you the mail with samples and expected output. please do the needful.

0 Karma

pratheep1980
New Member

@somesoni2 : the process outcome is derived based on around 10 attributes and they are being captured in the log along with all the transactions. All the Logs are getting indexed immediately. Multiple rules with multi values are configured in excel. The requirement is to compare the values of logs with excel and check the process outcome value is correct.

Rules sheet has multiple rows with multi values and they are expanding to 84 millions of rows.

0 Karma

somesoni2
Revered Legend

What's your requirement here (please provide some sample data/expected output)?

0 Karma

pratheep1980
New Member

The process outcome is derived based on around 10 attributes and they are being captured in the logs and indexed immediately. Multiple rules with multi values are configured in excel format. The requirement is to compare the values of logs with excel sheet and get the process outcome value from excel.
Getting 84 millions of rows after expanding the multi values of rules sheet.

0 Karma

pratheep1980
New Member

lookup csv file has multiple rows due to which makemv and mvexpand commands create millions of rows and the system has some 2 GB restrictions in storing the outputlookup files, hence the split file is not working.

0 Karma

niketnilay
Legend

@pratheep1980 you should move your lookup to KV Store. Please refer to the Splunk Dev site for instructions and also scenarios for Lookup File vs KV Store implementation: http://dev.splunk.com/view/webframework-developapps/SP-CAAAEZQ

____________________________________________
| makeresults | eval message= "Happy Splunking!!!"
0 Karma
Did you miss .conf21 Virtual?

Good news! The event's keynotes and many of its breakout sessions are now available online, and still totally FREE!