Hi
I have set up a data model and I am reading in millions of data lines.
The issue is some data lines are not displayed by tstats or perhaps the datamodel is not taking them in?
This is the query in tstats (2,503 events)
| tstats summariesonly=true count(All_TPS_Logs.duration) AS count FROM datamodel=MLC_TPS_DEBUG WHERE (nodename=All_TPS_Logs.User_Operations host=EXCESS_WORKFLOWS_UOB) GROUPBY All_TPS_Logs.operationIdentity
Result
All_TPS_Logs.fullyQualifiedMethod count
murex.limits.utilities.cache.statistics.CacheStatisticsTimerTask#publishCacheStatistics 2503
This is the same query in a normal search
index=mlc_live sourcetype=tps host=EXCESS_WORKFLOWS_UOB (user=* OR NOT user=*) NOT overflow=true | search name = "*" | eval fullyQualifiedMethod = name."#".operationIdentity |eval duration = endTime - startTime | stats count(duration) as Count by fullyQualifiedMethod
This is the result - We get more data! (As expected)
fullyQualifiedMethod Count
murex.limits.utilities.cache.statistics.CacheStatisticsTimerTask#publishCacheStatistics 2503
murex.risk.control.excesses.service.DefaultExcessService#closeAll 1
murex.risk.control.excesses.service.DefaultExcessService#enlist 408
murex.risk.control.excesses.service.DefaultExcessService#query 10
murex.risk.control.excesses.service.DefaultExcessService#transition 50672
*Data that both normal search and tstats are reading*
{"endTime":1474387803162,"startTime":1474387803162,"operationIdentity":"publishCacheStatistics","name":"murex.limits.utilities.cache.statistics.CacheStatisticsTimerTask","context":{"parentContext":{"id":-1,"parentContext":null},"data":[{"value":"BATCH-640@2016-04-13--CountryEERisk-164--ProfileNettingUTable-13067","key":"name"},{"value":"0","key":"hits"},{"value":"0","key":"misses"},{"value":"0","key":"count"},{"value":"4096","key":"maxElements"},{"value":"0","key":"evictions"},{"value":"Default","key":"policy"}],"id":1957}}
Example of Data not been picked up by Datamodel
{"endTime":1474384787832,"startTime":1474384787777,"operationIdentity":"closeAll","name":"murex.risk.control.excesses.service.DefaultExcessService"}
Another one
{"endTime":1474387506930,"startTime":1474387505531,"operationIdentity":"query","name":"murex.risk.control.excesses.service.DefaultExcessService"}
It looks like the large traces get picked up and the smaller ones dont...
Any help would be super 🙂 - Going crazy here.
... View more