Best approach using tstats for splunk dashboard and visualize data

[elend]

What is the best approach for data visualization using tstats? I am new to using tstats, I moved away from using the regular search index because it speeds up the query process.

for example making this query to show the vulnerabilities found on each ip

| tstats summariesonly=t dc(Vulnerability.signature) as vulnerabilities from datamodel=Vulnerability by Vulnerability.dest 
| sort -vulnerabilities 
| rename Vulnerability.dest as ip_address 
| table ip_address vulnerabilities

for example, first line from that query show ip 192.168.1.5 has 4521 vulnerabilities found


then I also created another detail table to verify and show some other columns related to that ip (click ip and send token) but it shows a different amount of data (4638 events).

| tstats summariesonly=t count FROM datamodel=Vulnerability WHERE Vulnerability.destination="192.168.1.5" AND Vulnerability.signature="*" BY Vulnerability.destination, Vulnerability.signature, Vulnerability.severity, Vulnerability.last_scan, Vulnerability.risk_score, Vulnerability.cve, Vulnerability.cvss_v3_score, Vulnerability.solution 
| `drop_dm_object_name(Vulnerability)` 
| rename destination as ip_address 
| fillnull value="Unknown" ip_address signature severity last_scan risk_score cve cvss_v3_score solution
| table ip_address signature severity last_scan risk_score cve cvss_v3_score solution

and I know this is related to the inaccuracy of the query, because if Ichange the "BY" parameter it will change the amount of data displayed too.

how to make the data count of this query match the same output as the first query, but still display other fields even though they are empty.

[gcusello]

Hi @elend ,

your two searches are completely different, so it's normal to have different results.

probably in the additional fields that you usend in the second search, there's some empty value, so for this reason the related results are discarded in the second search results.

In other words, you cannot compare these two searches.

to really compare them, you should modify the DataModel rules adding a calculated field that when there's an empty value for each field, it adds e fixed value (e.g.: "unknown"), as you can find for the user field in the authentication data model.

Ciao.

Giuseppe

[elend]

is it possible to make the null value filled with some value so it still counted?. i search for this option and there is some solution
- made change on props conf to eval the null value
- use tstats ... fillnull_value="null"

is there other option or best approach for this?

[gcusello]

Hi @elend ,

you are working on Datamodels, so the only approach is to creater a calculated field that, when the DM is populated, it takes a value when a field is empty, e.g.:

| eval destination=if(isempty(destination),"unknown",destination)

but you have to do this as a calculated field to use in the population searcjh, not in the same search.

Then you have to do this for all your fields.

Ciao.

Giuseppe

[elend]

Actually I already evals all field and made fillnull with "Unknonwn" strings all the fields. However some queries show same amount of event, but some field filled "Unknonwn" even it actually have values. 
Or rebuild the datamodel is needed?

[gcusello · Accepted solution]

Hi @elend ,

yes, you have to rebuild the DataModel, otherwise the change is applied only to new events.

Ciao.

Giuseppe