not picking up timestamp

[a212830]

Hi,

I added a feed (with some great help from this site), and it is working, except for the timestamp. Each line begins with an epoch timestamp, and I configured Splunk to recognize it, but it's picking up the time that is gets processed.

Input example:
1377468000000|139036|NormalizedMemoryInfo|Free|226005684|Memory|testhost|Enhanced-MemoryPool: Processor 1001.1
1377468000000|139036|NormalizedMemoryInfo|Utilization|29.881565745219028|Memory|testhost2|Enhanced-MemoryPool: Processor 1001.1

props.conf:
[snmpinfo]
MAX_TIMESTAMP_LOOKAHEAD = 30
SHOULD_LINEMERGE = FALSE
NO_BINARY_CHECK = 1
TIME_PREFIX = ^
LINE_BREAKER = ([\r\n]+)
TIME_FORMAT = %s
TRANSFORMS-set = setnull,setparsing

From what I read, the %s is for epoch time. Each event should have a 5 minute interval, but it's appearing with the timestamp when it got processed.

Also, it appears that the output mechanism doesn't always output a full line when writing. Is there any way to make sure it only pulls in entire events? I've seen half events show up.

[lcrielaa]

In order to make sure that Splunk only indexes full events, you can try 2 solutions.

One is to set the following value in your time_before_close in your inputs.conf:

time_before_close =
* Modtime delta required before Splunk can close a file on EOF.
* Tells the system not to close files that have been updated in past seconds.
* Defaults to 3.

This will tell Splunk to wait a certain amount of seconds before it assumes that the event has completed.
Another way is to use the LINE_BREAKER option in props.conf to tell Splunk what the end of a line of logging looks like. You'll have to be lucky enough that all your logfiles have a determinable ending. Remember that you can use PCRE here, so you can do a positive lookahead to the start of your next line as well to determine the ending of the previous line.

[sowings]

That's an epoch to milliseconds, you'll need TIME_FORMAT = %s%3N.