Hi,
I have a DBX input as follow:
[dbmon-tail://HPNA-DB/HPNA-Configs]
host = HPNA-DB
index = hpnaconfigs
output.format = mkv
output.timestamp = 1
output.timestamp.column = LastSnapshotSuccessDate
output.timestamp.format = yyyy-MM-dd HH:mm:ss.SSS
query = with Configs as (\r\n select p.PrimaryIPAddress DeviceIP\r\n ,p.hostname DeviceName\r\n ,p.LastSnapshotSuccessDate\r\n ,ConfigTextId = (select top 1 dd.DeviceDataId from RN_Device d inner join rn_device_data dd ON dd.DeviceID = d.DeviceID\r\n and d.DeviceID = p.deviceid \r\n and dd.BlockType = 'configuration' \r\n and dd.blockformat = 1\r\n order by dd.LastModifiedDate desc\r\n )\r\n from RN_DEVICE p\r\n)\r\nselect LastSnapshotSuccessDate\r\n ,DeviceName\r\n ,DeviceIP\r\n
,convert(varchar(50), dd.LastModifiedDate, 21) as LastModifiedDate\r\n
,substring(DataBlock,1,100) as ConfigTextStart\r\n
,substring(DataBlock,datalength(DataBlock)-100,100) as ConfigTextEnd\r\n
,datalength(DataBlock) as ConfigTextLen1\r\n ,DataBlock as ConfigText\r\n
,datalength(DataBlock) as ConfigTextLen2\r\nfrom Configs c inner join rn_device_data dd on dd.DeviceDataId = c.ConfigTextId\r\n{{WHERE $rising_column$ > ?}}
sourcetype = dbmon:mkv
tail.rising.column = LastSnapshotSuccessDate
disabled = 0
interval = auto
table = HPNA-Configs
and the following props.conf stanzas in system/local
, apps/dbx/local
and apps/search/local
:
[dbmon:mkv]
LINE_BREAKER_LOOKBEHIND = 100000
TRUNCATE = 0
MAX_EVENTS = 100000
However, when searching, events are being truncated after 10K.
Any idea?
I ran into a similar issue, was as if Splunk failed to honor the settings in props.conf. I ran across an answer (sorry can't find it now) that suggested using the tpl_*.dbmonevt source. It's solved my issue, could you try adding the following to your props.conf?
[source::...tpl_*.dbmonevt]
LINE_BREAKER_LOOKBEHIND = 100000
TRUNCATE = 0
MAX_EVENTS = 100000
Thanks for the suggestion, however it had no affect, the events are still capped at 10K exactly.
Note: the last column "ConfigTextLen2" in the query is never visible...
this is what the event is tag with as well:
host = HPNA-DB source = dbmon-tail://HPNA-DB/HPNA-Configs sourcetype = dbmon:mkv
A suggestion was made that if you are using the JDBC drivers that ship with DB Connect and this is MS SQL Server, to swap them out and use Drivers that are shipped directly from Microsoft.
Splunk indexer is running on Linux, i don't believe MS made an SQL driver for this OS.
yeah, they do make a Linux version -- you can get it here. http://www.microsoft.com/en-us/download/details.aspx?id=11774
I'm not positive that it's relevant to your problem, but we've found that it has fewer weirdnesses.
wow, that is a surprise. Maybe i should try it.
what type of database and which driver are you using?
I am using MS SQL Server and the Java driver that comes with Splunk