Is the rising column an integer that is auto generated by the table such as ROWID, etc? ... View more

Below are the differences between the two parameters: Max Rows to Retrieve: This will be the rows that will retrieved with each query. Note that it depends on how each client is implemented. For instance, PostgreSQL driver seems to honor this value and limits the data from DB, therefore both DB and DBX will be limited. However, other drivers may ignore this on the DB side (not like limiting the result by using LIMIT clause of SQL) and will honor it on DBX side only. Fetch Size: This value will be set to the JDBC driver as a hint on how many rows to be fetched one time from the database. Word "hint" is emphasized, since this also highly depends on how the database drivers are implemented. For example, Oracle drivers seems to honor this setting and try to fetch the rows in chunk, so that it can avoid from out of client side (DBX) memory errors, while some drivers might not honor it and just send the whole result in one big bulk. ... View more

Splunk checksums the file contents, as per the link ddrillic posted you can tweak the crcSalt, and the CHECK_METHOD, furthermore you can also tweak the initCrcLength in the inputs.conf In the example your providing above, I would expect Splunk to re-index the entire file as the file was overwritten, however it should be straightforward to test it... ... View more

The only data that will be parsed on an Universal or Lightweight forwarders (and all forwarders) will be the sourcetypes using INDEXED_EXTRACTIONS. that do tailing time structured data parsing. usually : xml, json, IIS, etc ... see http://docs.splunk.com/Documentation/Splunk/latest/Data/Extractfieldsfromfileswithstructureddata Check your data format and sourcetype. then if it is the case, you can prevent the errors by tuning your sourcetype parsing on the forwarder props.conf directly (like the MAX_EVENTS to raise to more than 256) ... View more

You need the unarchive_cmd option described in this blog post: http://blogs.splunk.com/2011/07/19/the-naughty-bits-how-to-splunk-binary-logfiles/ Alternatively, you can do a scripted input or a scripted lookup where you call another utility to convert to a plain text format (like HTML): http://docs.splunk.com/Documentation/Splunk/latest/AdvancedDev/ScriptedInputsIntro http://docs.splunk.com/Documentation/Splunk/6.4.1/Knowledge/Configureexternallookups Here are some that can help (I have NOT tried any of them): http://dbf-file-viewer.soft112.com/ http://dbfviewer.com/ ... View more

What REST API are you trying to query? Are you sure the response type is CSV? Can you send a sample response? Typically if Splunk is monitoring a CSV file there are all types of configurations you can use to parse that data in props.conf http://docs.splunk.com/Documentation/Splunk/latest/admin/Propsconf The following settings seem like they might help you out if you are in-fact monitoring CSV Data. INDEXED_EXTRACTIONS = < CSV|W3C|TSV|PSV|JSON > * Tells Splunk the type of file and the extraction and/or parsing method Splunk should use on the file. CSV - Comma separated value format TSV - Tab-separated value format PSV - pipe "|" separated value format W3C - W3C Extended Extended Log File Format JSON - JavaScript Object Notation format * These settings default the values of the remaining settings to the appropriate values for these known formats. * Defaults to unset. FIELD_NAMES = [ ,..., ] * Some CSV and structured files might have missing headers. This attribute tells Splunk to specify the header field names directly. TIMESTAMP_FIELDS = [ ,..., ] * Some CSV and structured files have their timestamp encompass multiple fields in the event separated by delimiters. This attribue tells Splunk to specify all such fields which constitute the timestamp in a comma-separated fashion. * If not specified, Splunk tries to automatically extract the timestamp of the event. ... View more

Hi, Reviewing the doc, the events are send to splunk via API, thats the reason that you have to configure the Chef application with the user,password and port 8089 of your splunk. Hope i help you. ... View more

This is done on the forwarder. Check out this Q&A: https://answers.splunk.com/answers/91386/the-limitation-of-monitor-files-in-one-directory.html Most people who put compressed files directly into inputs.conf will eventually see logs like this In splunkd.log : DateTime INFO TailingProcessor - failed to insert into AQ, retrying... This will be accompanied by logs like this in metrics.log : DateTime INFO Metrics - group=queue, name=aq, blocked=true, max_size=10000,filled_count=7, empty_count=0, current_size=10000, largest_size=10000, smallest_size=9996* Here AQ is the queue feeding the ArchiveProcessor , which is the thread that handles compressed and archived inputs (.gz, .bz2, .Z, .tar, .zip, .tgz). The ArchiveProcessor is single-threaded and handles archives one at a time. This means that the file processing code has found more than 10000 archive files that we are processing in turn. The only hope in this case is that the workload will have light periods to allow the ArchiveProcessor catch up on the backlog, but this may never happen. ... View more

While a far better solution would be fixing the log format, I wonder if a heavy forwarder could be employed to modify the date string before sending it to the indexing tier. If necessary, the forwarder could be configured to ignore the timestamp, it should simply apply the transformation, e.g. via SEDCMD, then send the data on. ... View more