apache_logs reporting no source

[crogersnlagovau]

Hi,

I've set up a central forwarder to send all our apache logs to the indexes. Here is one of the monitor sections:

[monitor:///weblogs/catalogue/]
sourcetype = access_combined
whitelist=access_log.*[0-9].gz
index = apache_blah
host = apacheblah.com
ignoreOlderThan = 31d
recursive = true

However, I'm seeing this error in splunkd.log on the forwarder:

WARN  TcpOutputProc - The event is missing source information. Event :

When I look at the source field, it is reporting the log filename. This is affecting me using Traffic Ray because the host information is using the source field.

How can I get the source section saying apache_blah.com or catalogue?

Regards

[alacercogitatus]

Typically, you don't want to consume GZ files, as they contain LARGE amounts of data and is not RealTime. I'd suggest consuming the access_log in the raw form first, it will be continually monitored and you won't have to worry about GZ rotations.

As for the Traffic Ray App, the way the data input is crafted is not a Best Practice. However, you should be able to fix it by REMOVING the REGEX on a path for the host field. This way, your input will use the specified apacheblah.com setting, and not something from the path.

[woodcock]

It seems OK to me but give this alternate syntax a try:

[monitor:///weblogs/catalogue/.../access_log.*[0-9].gz]
sourcetype = access_combined
index = apache_blah
host = apacheblah.com
ignoreOlderThan = 31d
recursive = true