topic Exclude records from the final result in Getting Data In

Exclude records from the final result

Anand_Raman — Fri, 29 Jan 2021 18:11:24 GMT

Hi, I have a situation where I have to exclude certain records from the final result only. I don't want to include in the initial search criteria because it excludes the whole event. For example a single event may contain many tables and if I exclude a certain table in my initial search I'm getting a wrong count of other tables . I just want those table removed from my final results.

Thanks

Re: Exclude records from the final result

isoutamo — Fri, 29 Jan 2021 19:31:29 GMT

usually this can do wit search and/or where commands. If you give some example data and full SPL query, we could help you with it.

r. Ismo

Re: Exclude records from the final result

Anand_Raman — Fri, 29 Jan 2021 20:07:54 GMT

Thanks Soutamo for the reply. The below is my main search

"index="production_index" host="production" source="projects/production/logs/cloudaudit.googleapis.com%2Fdata_access" "protoPayload.authenticationInfo.principalEmail" = "*@.com"
protoPayload.serviceData.jobGetQueryResultsResponse.job.jobStatistics.referencedTables{}.datasetId!="_*"
protoPayload.serviceData.jobGetQueryResultsResponse.job.jobStatistics.createTime=*"

I'm parsing GCP logs to find number of users per dataset. One SQL may contain reference to multiple datasets and when I exclude the control tables in my search the whole event is being excluded.