Is it possible to use dedup or "|" commands in the base search of a data model?

[bwindham]

I have an instance using ServiceNow data where I want to dedup the data based on sys_updated_on to get the last update and status of the incident.

sourcetype="snow:pm_project" | dedup number sortby -sys_updated_on

However, I keep getting "|" pipes are not allowed.

I started looking at modifying the data model json file, but still got the message.

I might note that I am using "Root Event" to get acceleration to work with this.
I know I can do searches to use dedup. Should I use "Root Search" and "Root Event" together? Not sure how I would do that.

This is the search I ultimately want:

sourcetype="snow:pm_project" | dedup number sortby -sys_updated_on | search state=-5   

where state would be child objects beneath the base search.

Any help would be appreciated.....I have spent a lot of time banging my head on this and want to use data models for acceleration.

[tskinnerivsec · Accepted solution]

A base search is usually just that, very basic "sourcetype = . . . " I've used a number of Data Models that have duplicate values in their fields, take for instance the Email data model, depending on how many sourcetypes are feeding it and from where, you can easily have duplicate messageid vailues. I have never needed to solve this problem within the data model, I've always been able to dedup after the data model search. For example:

| tstats summariesonly=t allow_old_summaries=t sum(All_Email.size) as "EmailMessageSize", from datamodel=Email where All_Email.user = $user$ groupby All_Email.user _time All_Email.subject All_Email.message_id All_Email.recipient span=1s | search All_Email.recipient != *foo* | dedup All_Email.message_id | . . .

[bwindham]

Thanks tskinnerivesec....I'll go that route.