SPL to retrieve more fields

[indut]

Hi all,

I am using splunk after a while and lost touch with the SPL. Please help me on below.

I have about 40 fields to extract using a SPL query. I am able to get all the fields required using interesting fields.

The issue that I am facing is that I am getting duplicate records in my result set (possibly it is due to the multiple source types that I am using in my query).  Just wondering what is the correct way to write SPL so that all fields that I retrieve are unique records. Don't think writing dedup on all 40 fields is a good idea. Also not sure if I use stats function,  do I have to write values(empno) as empno, vaues(empstartdate) as startdate.........on all 40 fields ? (If my data set has all employee details as an example)

Thanks in advance!

[somesoni2 · Accepted solution]

Do all 40 fields appear in every event? If yes, then use the suggested dedup command below.

If all 40 fields do not appear in every event, do you have a primary key (field or fields that uniquely identify your event)? If yes, then your can do something like this.

your base search | fields ...list all 40 fields...
| stats values(*) as * by primary_key_fields

In all other cases it would be very difficult to get unique records, without multiple stats/streamstats.

[ITWhisperer]

You could try just dedup _raw