https://community.splunk.com/t5/Dashboards-Visualizations/Duplicate-values-for-each-fields-coming-up-where-lookup-is-used/m-p/647611#M52961 <P>index=standard_tanium source="Prod-CSV" script=read.ps1"<BR />|rename field01 as user<BR />|rename field02 as login-count<BR />|rename CSV-Timestamp as Script_run<BR />|rex mode=sed field=user "/[&amp;?].*//g"<BR />|table Script_run IDBC-Hostname user login-count<BR /><STRONG><BR />|lookup read-login.csv user-id as user OUTPUTNEW user-id<BR />|search user-id=*<BR />|sort -login-count<BR />|fields Script_run IDBC-Hostname user login-count tier sownername servicename<BR /><BR />This gives output as below--<BR /><BR /></STRONG></P><P>Script_run &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; IDBC-Hostname&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp; user&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp; login-count&nbsp;&nbsp;&nbsp;&nbsp;&nbsp; tier&nbsp;&nbsp;&nbsp;&nbsp; sownername&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp; servicename</P><P>2023/06/19 12:25&nbsp;&nbsp;&nbsp;&nbsp; abc12345&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp; 11133121-990&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp; 4<BR />2023/06/15 17:25&nbsp;&nbsp;&nbsp;&nbsp; xzyz1112&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp; 33421111-990&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp; 2</P><P>2023/06/14 19:15&nbsp;&nbsp;&nbsp;&nbsp; arsd1123&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp; 7877333-750&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp; 1</P><P>2023/06/12 12:25&nbsp;&nbsp;&nbsp;&nbsp;&nbsp; abc12345&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp; 9911128-630&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp; 3</P><P><STRONG>&nbsp;</STRONG></P> Tue, 20 Jun 2023 13:43:43 GMT srv007 2023-06-20T13:43:43Z https://community.splunk.com/t5/Dashboards-Visualizations/Duplicate-values-for-each-fields-coming-up-where-lookup-is-used/m-p/647520#M52943 <P>Below is the splunk query that i'm using.<BR /><BR /></P> <LI-CODE lang="markup">index=standard_tanium source="Prod-CSV" script=read.ps1" |rename field01 as user |rename field02 as login-count |rename CSV-Timestamp as Script_run |rex mode=sed field=user "/[&amp;?].*//g" |table Script_run IDBC-Hostname user login-count |lookup read-login.csv user-id as user OUTPUTNEW user-id |lookup owner.csv hostname as IDBC-Hostname OUTPUTNEW tier sownername servicename |search user-id=* |sort - login-count | fields Script_run IDBC-Hostname user login-count tier sownername servicename</LI-CODE> <P><BR /><BR />the above query is working fine, just duplicate values coming for <STRONG>tier sownername servicename<BR /><BR />Output :<BR /></STRONG><BR />Script_run&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp; IDBC-Hostname&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp; user&nbsp;&nbsp;&nbsp;&nbsp;&nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; login-count&nbsp;&nbsp;&nbsp;&nbsp; tier sownername servicename<BR /><BR />2023/06/19 12:25&nbsp;&nbsp;&nbsp; abc12345&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp; 11133121-990&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp; 4&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp; 2&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp; Bob&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp; ADF-Co</P> <P>&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp; 2&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp; Bob&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp; ADF-Co</P> <P>&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp; 2&nbsp;&nbsp;&nbsp;&nbsp;&nbsp; Bob&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp; ADF-Co</P> <P>&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp; 2&nbsp;&nbsp;&nbsp;&nbsp; Bob&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp; ADF-Co</P> <P>2023/06/15 17:25&nbsp;&nbsp;&nbsp; xzyz1112&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp; 33421111-990&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp; 2&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp; 1&nbsp;&nbsp;&nbsp; Sam&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp; AXF-Co</P> <P>&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp; 1&nbsp;&nbsp; Sam&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp; AXF-Co</P> <P>&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp; 1&nbsp; Sam&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp; AXF-Co</P> <P>&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp; 1 Sam&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp; AXF-Co</P> Tue, 20 Jun 2023 13:23:21 GMT https://community.splunk.com/t5/Dashboards-Visualizations/Duplicate-values-for-each-fields-coming-up-where-lookup-is-used/m-p/647520#M52943 srv007 2023-06-20T13:23:21Z https://community.splunk.com/t5/Dashboards-Visualizations/Duplicate-values-for-each-fields-coming-up-where-lookup-is-used/m-p/647577#M52951 <P>want to understand why the lookup fields are appearing repeated and how to fix it.<BR />I did try<BR />|mvexpand tier | dedup IDBC-Hostname<BR /><BR />but then that removes duplicate entries which shouldn't happen as it can be case same server repeated and we need data for it. This dedup affects login-count since dedup removes duplicate hostnames.</P> Tue, 20 Jun 2023 11:09:34 GMT https://community.splunk.com/t5/Dashboards-Visualizations/Duplicate-values-for-each-fields-coming-up-where-lookup-is-used/m-p/647577#M52951 srv007 2023-06-20T11:09:34Z https://community.splunk.com/t5/Dashboards-Visualizations/Duplicate-values-for-each-fields-coming-up-where-lookup-is-used/m-p/647584#M52952 <P>Try mvdedup(user)</P><LI-CODE lang="markup">index=standard_tanium source="Prod-CSV" script=read.ps1" |rename field01 as user |rename field02 as login-count |rename CSV-Timestamp as Script_run |rex mode=sed field=user "/[&amp;?].*//g" |table Script_run IDBC-Hostname user login-count | eval user=mvdedup(user) |lookup read-login.csv user-id as user OUTPUTNEW user-id |lookup owner.csv hostname as IDBC-Hostname OUTPUTNEW tier sownername servicename |search user-id=* |sort - login-count | fields Script_run IDBC-Hostname user login-count tier sownername servicename</LI-CODE> Tue, 20 Jun 2023 11:51:31 GMT https://community.splunk.com/t5/Dashboards-Visualizations/Duplicate-values-for-each-fields-coming-up-where-lookup-is-used/m-p/647584#M52952 ITWhisperer 2023-06-20T11:51:31Z https://community.splunk.com/t5/Dashboards-Visualizations/Duplicate-values-for-each-fields-coming-up-where-lookup-is-used/m-p/647592#M52953 <P>still same result, no changes</P> Tue, 20 Jun 2023 12:08:58 GMT https://community.splunk.com/t5/Dashboards-Visualizations/Duplicate-values-for-each-fields-coming-up-where-lookup-is-used/m-p/647592#M52953 srv007 2023-06-20T12:08:58Z https://community.splunk.com/t5/Dashboards-Visualizations/Duplicate-values-for-each-fields-coming-up-where-lookup-is-used/m-p/647593#M52954 <P>Try the same with&nbsp;IDBC-Hostname</P><LI-CODE lang="markup">index=standard_tanium source="Prod-CSV" script=read.ps1" |rename field01 as user |rename field02 as login-count |rename CSV-Timestamp as Script_run |rex mode=sed field=user "/[&amp;?].*//g" |table Script_run IDBC-Hostname user login-count | eval user=mvdedup(user) |lookup read-login.csv user-id as user OUTPUTNEW user-id | eval IDBC-Hostname=mvdedup('IDBC-Hostname') |lookup owner.csv hostname as IDBC-Hostname OUTPUTNEW tier sownername servicename |search user-id=* |sort - login-count | fields Script_run IDBC-Hostname user login-count tier sownername servicename</LI-CODE> Tue, 20 Jun 2023 12:12:13 GMT https://community.splunk.com/t5/Dashboards-Visualizations/Duplicate-values-for-each-fields-coming-up-where-lookup-is-used/m-p/647593#M52954 ITWhisperer 2023-06-20T12:12:13Z https://community.splunk.com/t5/Dashboards-Visualizations/Duplicate-values-for-each-fields-coming-up-where-lookup-is-used/m-p/647594#M52955 <P>still same, no help.<BR /><BR /></P><P>&nbsp;</P> Tue, 20 Jun 2023 12:16:48 GMT https://community.splunk.com/t5/Dashboards-Visualizations/Duplicate-values-for-each-fields-coming-up-where-lookup-is-used/m-p/647594#M52955 srv007 2023-06-20T12:16:48Z https://community.splunk.com/t5/Dashboards-Visualizations/Duplicate-values-for-each-fields-coming-up-where-lookup-is-used/m-p/647596#M52956 <P>Perhaps it is time for you to share some of your events. For example, what do you get after this part of the search?</P><LI-CODE lang="markup">index=standard_tanium source="Prod-CSV" script=read.ps1" |rename field01 as user |rename field02 as login-count |rename CSV-Timestamp as Script_run |rex mode=sed field=user "/[&amp;?].*//g" |table Script_run IDBC-Hostname user login-count</LI-CODE> Tue, 20 Jun 2023 12:20:14 GMT https://community.splunk.com/t5/Dashboards-Visualizations/Duplicate-values-for-each-fields-coming-up-where-lookup-is-used/m-p/647596#M52956 ITWhisperer 2023-06-20T12:20:14Z https://community.splunk.com/t5/Dashboards-Visualizations/Duplicate-values-for-each-fields-coming-up-where-lookup-is-used/m-p/647604#M52958 <P class="lia-align-left"><STRONG>Below is the result :</STRONG></P><P class="lia-align-left">&nbsp;</P><P>Script_run&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp; IDBC-Hostname&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp; user&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp; login-count</P><P>2023/06/19 12:25&nbsp;&nbsp;&nbsp; abc12345&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;11133121-990 &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp; 4<BR />2023/06/15 17:25&nbsp;&nbsp;&nbsp; xzyz1112&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp; 33421111-990&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp; 2</P><P>2023/06/14 19:15&nbsp;&nbsp;&nbsp; arsd1123&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp; 7877333-750&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp; 1</P><P>2023/06/12 12:25&nbsp;&nbsp;&nbsp; abc12345&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp; 9911128-630&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp; 3<BR /><BR /><BR />Here it is expected to have repeated hostnames as&nbsp; the user id is varying and we need sum total of login-count so we shouldnt dedup hostnames.<BR /><BR />Now the problem of duplicate value happens for fields which we using for lookup like tier</P><P>&nbsp;</P> Tue, 20 Jun 2023 12:47:41 GMT https://community.splunk.com/t5/Dashboards-Visualizations/Duplicate-values-for-each-fields-coming-up-where-lookup-is-used/m-p/647604#M52958 srv007 2023-06-20T12:47:41Z https://community.splunk.com/t5/Dashboards-Visualizations/Duplicate-values-for-each-fields-coming-up-where-lookup-is-used/m-p/647609#M52960 <P>Try adding one more line at a time to find out which line is producing the duplicates</P> Tue, 20 Jun 2023 13:29:20 GMT https://community.splunk.com/t5/Dashboards-Visualizations/Duplicate-values-for-each-fields-coming-up-where-lookup-is-used/m-p/647609#M52960 ITWhisperer 2023-06-20T13:29:20Z https://community.splunk.com/t5/Dashboards-Visualizations/Duplicate-values-for-each-fields-coming-up-where-lookup-is-used/m-p/647611#M52961 <P>index=standard_tanium source="Prod-CSV" script=read.ps1"<BR />|rename field01 as user<BR />|rename field02 as login-count<BR />|rename CSV-Timestamp as Script_run<BR />|rex mode=sed field=user "/[&amp;?].*//g"<BR />|table Script_run IDBC-Hostname user login-count<BR /><STRONG><BR />|lookup read-login.csv user-id as user OUTPUTNEW user-id<BR />|search user-id=*<BR />|sort -login-count<BR />|fields Script_run IDBC-Hostname user login-count tier sownername servicename<BR /><BR />This gives output as below--<BR /><BR /></STRONG></P><P>Script_run &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; IDBC-Hostname&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp; user&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp; login-count&nbsp;&nbsp;&nbsp;&nbsp;&nbsp; tier&nbsp;&nbsp;&nbsp;&nbsp; sownername&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp; servicename</P><P>2023/06/19 12:25&nbsp;&nbsp;&nbsp;&nbsp; abc12345&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp; 11133121-990&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp; 4<BR />2023/06/15 17:25&nbsp;&nbsp;&nbsp;&nbsp; xzyz1112&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp; 33421111-990&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp; 2</P><P>2023/06/14 19:15&nbsp;&nbsp;&nbsp;&nbsp; arsd1123&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp; 7877333-750&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp; 1</P><P>2023/06/12 12:25&nbsp;&nbsp;&nbsp;&nbsp;&nbsp; abc12345&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp; 9911128-630&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp; 3</P><P><STRONG>&nbsp;</STRONG></P> Tue, 20 Jun 2023 13:43:43 GMT https://community.splunk.com/t5/Dashboards-Visualizations/Duplicate-values-for-each-fields-coming-up-where-lookup-is-used/m-p/647611#M52961 srv007 2023-06-20T13:43:43Z https://community.splunk.com/t5/Dashboards-Visualizations/Duplicate-values-for-each-fields-coming-up-where-lookup-is-used/m-p/647613#M52962 <P>index=standard_tanium source="Prod-CSV" script=read.ps1"<BR />|rename field01 as user<BR />|rename field02 as login-count<BR />|rename CSV-Timestamp as Script_run<BR />|rex mode=sed field=user "/[&amp;?].*//g"<BR />|table Script_run IDBC-Hostname user login-count</P><P>|lookup read-login.csv user-id as user OUTPUTNEW user-id<BR /><STRONG><BR />|lookup owner.csv hostname as IDBC-Hostname OUTPUTNEW tier sownername servicename<BR /><BR /></STRONG>|search user-id=*<BR />|sort -login-count<BR />|fields Script_run IDBC-Hostname user login-count tier sownername servicename<STRONG><BR /><BR /><BR />When i add lookup for owner.csv then its creating duplicates for the lookup fields.<BR /><BR />owner.csv has </STRONG>tier,sownername servicename field values which i'm comparing and adding to my main search<BR /><STRONG>read-login.csv </STRONG>has user account info.<BR /><BR />read.ps1 has info about about all hostnames and login count. Using read-login.csv im fetching details for accounts mentioned in that csv only, and doing lookup with <STRONG>owner.csv</STRONG> i'm adding the extra fields to the search results.</P> Tue, 20 Jun 2023 13:50:41 GMT https://community.splunk.com/t5/Dashboards-Visualizations/Duplicate-values-for-each-fields-coming-up-where-lookup-is-used/m-p/647613#M52962 srv007 2023-06-20T13:50:41Z https://community.splunk.com/t5/Dashboards-Visualizations/Duplicate-values-for-each-fields-coming-up-where-lookup-is-used/m-p/647616#M52963 <P>So each event has one value for IDBC-Hostname, and your owner.csv has only one entry for each hostname?</P> Tue, 20 Jun 2023 13:55:53 GMT https://community.splunk.com/t5/Dashboards-Visualizations/Duplicate-values-for-each-fields-coming-up-where-lookup-is-used/m-p/647616#M52963 ITWhisperer 2023-06-20T13:55:53Z https://community.splunk.com/t5/Dashboards-Visualizations/Duplicate-values-for-each-fields-coming-up-where-lookup-is-used/m-p/647617#M52964 <P>It looks like you have multiple entries in your owner.csv for hostname.</P><P>Ideally, you should only have one entry</P><P>If you don't want to resolve the duplicates in your owner.csv file, try something like this</P><LI-CODE lang="markup">|stats values(*) as * by Script_run IDBC-Hostname user login-count</LI-CODE> Tue, 20 Jun 2023 14:03:06 GMT https://community.splunk.com/t5/Dashboards-Visualizations/Duplicate-values-for-each-fields-coming-up-where-lookup-is-used/m-p/647617#M52964 ITWhisperer 2023-06-20T14:03:06Z https://community.splunk.com/t5/Dashboards-Visualizations/Duplicate-values-for-each-fields-coming-up-where-lookup-is-used/m-p/647621#M52965 <P>owner.csv has only one entry for each hostname?&nbsp; &gt;&gt; correct<BR />So each event has one value for IDBC-Hostname&nbsp; &gt;&gt; can have more, hostname can re-appear but against different user and its run daily, so can have more than one entry for same hostname.</P> Tue, 20 Jun 2023 14:23:55 GMT https://community.splunk.com/t5/Dashboards-Visualizations/Duplicate-values-for-each-fields-coming-up-where-lookup-is-used/m-p/647621#M52965 srv007 2023-06-20T14:23:55Z https://community.splunk.com/t5/Dashboards-Visualizations/Duplicate-values-for-each-fields-coming-up-where-lookup-is-used/m-p/647627#M52967 <P>Thanks Mate, this did help.<BR />all repeated fields are gone, at the same time not affecting the hostname.<BR />so output is below :<BR /><BR /></P><P><span class="lia-inline-image-display-wrapper lia-image-align-inline" image-alt="srv007_0-1687271778521.png" style="width: 724px;"><img src="https://community.splunk.com/t5/image/serverpage/image-id/25921i3C82F1C728C56B4E/image-dimensions/724x132?v=v2" width="724" height="132" role="button" title="srv007_0-1687271778521.png" alt="srv007_0-1687271778521.png" /></span><BR />The only concern is can we remove user-id from getting displayed as same info already being shown under user.<BR />And if we can re-allign the order of the fields.<BR />example --<BR /><BR /><STRONG>Script_run&nbsp; IDBC-Hostname user login-count tier sownername servicename</STRONG></P><P>&nbsp;</P> Tue, 20 Jun 2023 14:38:17 GMT https://community.splunk.com/t5/Dashboards-Visualizations/Duplicate-values-for-each-fields-coming-up-where-lookup-is-used/m-p/647627#M52967 srv007 2023-06-20T14:38:17Z https://community.splunk.com/t5/Dashboards-Visualizations/Duplicate-values-for-each-fields-coming-up-where-lookup-is-used/m-p/647629#M52968 <P>Where did you add the stats command?</P> Tue, 20 Jun 2023 14:42:01 GMT https://community.splunk.com/t5/Dashboards-Visualizations/Duplicate-values-for-each-fields-coming-up-where-lookup-is-used/m-p/647629#M52968 ITWhisperer 2023-06-20T14:42:01Z https://community.splunk.com/t5/Dashboards-Visualizations/Duplicate-values-for-each-fields-coming-up-where-lookup-is-used/m-p/647630#M52969 <P>Update --<BR /><BR />i added <STRONG>| table </STRONG>and the fields to show and this as fixed the above mentioned too.</P><P>Kudos!!</P> Tue, 20 Jun 2023 14:42:05 GMT https://community.splunk.com/t5/Dashboards-Visualizations/Duplicate-values-for-each-fields-coming-up-where-lookup-is-used/m-p/647630#M52969 srv007 2023-06-20T14:42:05Z https://community.splunk.com/t5/Dashboards-Visualizations/Duplicate-values-for-each-fields-coming-up-where-lookup-is-used/m-p/647632#M52970 <P>index=standard_tanium source="Prod-CSV" script=read.ps1"<BR />|rename field01 as user<BR />|rename field02 as login-count<BR />|rename CSV-Timestamp as Script_run<BR />|rex mode=sed field=user "/[&amp;?].*//g"<BR />|table Script_run IDBC-Hostname user login-count<BR />|lookup read-login.csv user-id as user OUTPUTNEW user-id<BR />|lookup owner.csv hostname as IDBC-Hostname OUTPUTNEW tier sownername servicename<STRONG><BR /></STRONG>|search user-id=*<BR />|sort -login-count<BR /><FONT color="#339966"><STRONG>|stats values(*) as * by Script_run&nbsp;IDBC-Hostname user login-count</STRONG></FONT><BR /><FONT color="#339966">|table Script_run IDBC-Hostname user login-count tier sownername servicename</FONT><STRONG><BR /></STRONG></P> Tue, 20 Jun 2023 14:50:34 GMT https://community.splunk.com/t5/Dashboards-Visualizations/Duplicate-values-for-each-fields-coming-up-where-lookup-is-used/m-p/647632#M52970 srv007 2023-06-20T14:50:34Z