@bowesmana I am using two different indexes and joining using host as common field ..The first index is preos which has events as below for each host .Below preo1 is the host 2022-10-06T06:26:22.075037-07:00 preo1 log-inventory.sh[24540]: Boot timestamp: 2022-10-06 06:15:43 2022-10-06T06:26:22.074872-07:00 preo1 log-inventory.sh[24540]: G driver: version: 55.16, md5sum: B67829A0 2022-10-06T06:26:22.073636-07:00 preo1 log-inventory.sh[24540]: GP: PCISLOT: 000:5:00.0, MODEL: H0 e, PN: 2336-884-A1, SN: 133120196, VBIOS: 6.0.A.00, INFOROM: 520.01.01. I am using the below query using the above index index=preos host=preo1 ("Boot timestamp" OR "GPU driver: version: " OR *0000:45:00*)
| rex field=_raw "sh\[\d*\]\:\s*(?<GPU>[^\:]+)\s*\:\s*PCISLOT\:\s*(?<PCIe_Bus_Id>[^\,]+)\,\s*\w*\:\s.*\PN\:(?<PN>[^\,]+)\,\s*SN\:\s*(?<SN>[^.\,]+)\,\s*VBIOS\:\s*(?<VBIOS>[^\,]+)"
| rex field=_raw "Boot\s*timestamp\:\s*(?<Boot_Time>[^.*]+)"
| rex field=_raw "log\-inventory\.sh\[(?<fru_log_id>[^\]]+)"
| rex field=PCIe_Bus_Id "0000(?<PCIe_Bus_Id>[^\.\0]+)"
| rex field=_raw "GPU\sdriver\:\sversion\:\s(?<NV_Driver>[^\,]+)\,"
| stats values(Boot_Time) as Last_Boot_Time values(SN) as SN VALUES(PN) AS PN VALUES(VBIOS) AS NV_VBIOS VALUES(NV_Driver) AS NV_DRIVER values(PCIe_Bus_Id) as PCIe_Bus_Id BY fru_log_id host
| fillnull value=clear
| search SN!=clear PN!=clear NV_VBIOS!=clear NV_DRIVER!=clear
| fields host fru_log_id Last_Boot_Time PCIe_Bus_Id NV_VBIOS NV_DRIVER PN SN The second index is index=syslog with for the same host preo1.Below is the sample event 2022-10-03T05:20:41.545640-07:00 preo1 kernel: [24052.421284] NVRM: Xid (PCI:0000:85:00): 74, pid='<unknown>', name=<unknown>, NVLink: fatal error detected on link 6(0x10000, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0) Below is the query I am using index=syslog "*NVRM: Xid " process=kernel host IN (preo1)
| rex field=_raw "NVRM\:\sXid\s*\(PCI\:(?<PCIe_Bus_Id>[^ ]+)\)\:\s*(?<Error_Code>[^ ]+)\,\spid\=(?<pid>[^ ]+)\,\s*name\=(?<name>[^ ]+)\,\s(?<Log_Message>.*)"
| search Error_Code="***"
| stats count by host _time PCIe_Bus_Id Error_Code pid name Log_Message
| addcoltotals count labelfield=host label=_Total
| rename _time as eror_time
| table eror_time host PCIe_Bus_Id Error_Code pid name Log_Message count
| sort by -log_time
| dedup pid Now from the above two searches .I am trying to join by host using below search index=preos_inventory host=preos0036 ("Boot timestamp" OR "GPU driver: version: " OR *0000:45:00*)
| rex field=_raw "sh\[\d*\]\:\s*(?<GPU>[^\:]+)\s*\:\s*PCISLOT\:\s*(?<PCIe_Bus_Id>[^\,]+)\,\s*\w*\:\s.*\PN\:(?<PN>[^\,]+)\,\s*SN\:\s*(?<SN>[^.\,]+)\,\s*VBIOS\:\s*(?<VBIOS>[^\,]+)"
| rex field=_raw "Boot\s*timestamp\:\s*(?<Boot_Time>[^.*]+)"
| rex field=_raw "log\-inventory\.sh\[(?<fru_log_id>[^\]]+)"
| rex field=PCIe_Bus_Id "0000(?<PCIe_Bus_Id>[^\.\0]+)"
| rex field=_raw "GPU\sdriver\:\sversion\:\s(?<NV_Driver>[^\,]+)\,"
| stats values(Boot_Time) as Last_Boot_Time values(SN) as SN VALUES(PN) AS PN VALUES(VBIOS) AS NV_VBIOS VALUES(NV_Driver) AS NV_DRIVER values(PCIe_Bus_Id) as PCIe_Bus_Id BY fru_log_id host
| fillnull value=clear
| search SN!=clear PN!=clear NV_VBIOS!=clear NV_DRIVER!=clear
| fields host fru_log_id Last_Boot_Time PCIe_Bus_Id NV_VBIOS NV_DRIVER PN SN
| sort Last_Boot_Time
| join host
[search index=syslog "*NVRM: Xid " process=kernel host IN (preos0036)
| rex field=_raw "NVRM\:\sXid\s*\(PCI\:(?<PCIe_Bus_Id>[^ ]+)\)\:\s*(?<Error_Code>[^ ]+)\,\spid\=(?<pid>[^ ]+)\,\s*name\=(?<name>[^ ]+)\,\s(?<Log_Message>.*)"
| search Error_Code="***"
| stats count by host _time PCIe_Bus_Id Error_Code pid name Log_Message
| addcoltotals count labelfield=host label=_Total
| rename _time as eror_time
| table eror_time host PCIe_Bus_Id Error_Code pid name Log_Message count
| sort by -log_time
| dedup pid]
| eval eror_time=strftime(eror_time,"%Y-%m-%d %H:%M:%S")
| eval diff=eror_time-Last_Boot_Time But the issue is when I am trying to join .I see all results..no issues ..but when I am trying to get the difference between Last_Boot_Time and eror_time I dont get any results.I hope this is clear with events
... View more