Getting Data In

Can you set a certain time forwarding occurs?

aywong
Path Finder

How often does a forwarder check its logs and forward data?
Can I set some sort of configuration where forwarders only forward data at lets say 12:00 AM at night?

Tags (1)
0 Karma
1 Solution

Ayn
Legend

bmacias84
Champion

The easiest way to accomplish this would be a scripted input with Splunk's CLI "add one shotmore" command or with the type/cat command. My example to does not have any logic to handle datetime appends to file names.

Additional info:

  • 35http://docs.splunk.com/Documentation/Splunk/4.3.3/Data/MonitorfilesanddirectoriesusingtheCLI35

  • 35http://docs.splunk.com/Documentation/Splunk/4.3.1/Developer/ScriptedInputsIntro35

  • On your forwarer edit you inputs.conf

    #windows
    [script://.daily_file.bat c:\somedir\somelog.log]
    disabled = true
    index = main
    source = c:\somedir\somelog.log
    sourcetype = ras
    interval = 0 0 * * *

    #*nix
    [script://./bin/daily_file.sh /somedir/somelog.log]
    disabled = true
    index = main
    source = /somedir/somelog.log
    sourcetype = ras
    interval = 0 0 * * *

    Option 1


    #*nix
    #!/bin/bash
    $SPLUNK_HOME/bin/splunk add oneshot $1 -auth admin:changeme
    done


    #windows
    echo off
    %splunk%\splunk add oneshot %1 -auth admin:changeme

    Option 2
    This will read the output from cat or type which would normally be displayed on your CMD prompt.


    #*nix
    #!/bin/bash
    cat $1
    done


    #windows
    echo off
    type %1

    Hope this helps you.

    aywong
    Path Finder

    Thanks this helps!

    0 Karma

    bmacias84
    Champion

    All scripted inputs are ran on the cron notation.

    jguarini
    Path Finder

    for clarification Option1 is a script ran by cron or task manager, while Option 2 is a scripted input in Splunk.

    also note that there should be two additional * for the cron entry

    ie. 0 0 * * *

    bmacias84
    Champion

    @aywong, Why would you want to collect at midnight, can you give your use case?

    Ayn is correct there is not a way directly built into splunk for exact time based forwarding of data, but there are ways to accomplish this. Most monitoring done on a poll interval.

    File monitoring is based on file checksum. I belive it isendpoint_md5 by default.

    0 Karma

    Ayn
    Legend

    Ayn
    Legend

    The forwarder sends data as soon as it has any to send. How often it CHECKS data depends entirely on which type of input you're talking about. Some inputs, like scripted inputs and WMI inputs, will run at certain configurable intervals. Other inputs, like a regular file monitor, will check on such a regular basis that you can consider it to be more or less real-time.

    aywong
    Path Finder

    okay thanks, do you know how often a forwarder checks and forwards data though?

    0 Karma
    .conf21 Now Fully Virtual!
    Register for FREE Today!

    We've made .conf21 totally virtual and totally FREE! Our completely online experience will run from 10/19 through 10/20 with some additional events, too!