Getting Data In

Timestamp configuration for different sourcetypes of the same source

Path Finder

Hi, I have used props.conf and transforms.conf to configure two different sourcetypes coming to Splunk from udp:514. And when I tried to set timestamp extraction for these two sourcetypes I can't get it to work. Anything I overlooked? Thanks.

Splunk 4.1.4 installed on CentOS 5.2 32bit

props.conf -

[source::udp:514]
TRANSFORMS-UDP514Sourcetyping = SLB_AlteonOS,Juniper_VPN

[SLB_AlteonOS]
TIME_PREFIX = \w+\s+\d+\s+\d+:\d+:\d+\s+\d+\.\d+\.\d+\.\d+\s+
TIME_FORMAT = %b %d %H:%M:%S
MAX_TIMESTAMP_LOOKAHEAD = 49
priority = 101

[Juniper_VPN]
TIME_PREFIX = Juniper:\s+
TIME_FORMAT = %Y-%m-%d %H:%M:%S
MAX_TIMESTAMP_LOOKAHEAD = 62
priority = 101

transforms.conf -

[SLB_AlteonOS]
REGEX = ^\w+\s+\d+\s+\d+:\d+:\d+\s+\d+\.\d+\.\d+\.\d+\s+\w+\s+\d+\s+\d+:\d+:\d+\s+([\w\d-]+\s)?\w+\s+AlteonOS\s+
DEST_KEY = MetaData:Sourcetype
FORMAT = sourcetype::SLB_AlteonOS

[Juniper_VPN]
REGEX = ^\w+\s+\d+\s+\d+:\d+:\d+\s+10\.10\.8\.2\s
DEST_KEY = MetaData:Sourcetype
FORMAT = sourcetype::Juniper_VPN
0 Karma
1 Solution

Splunk Employee
Splunk Employee

You can't configure timestamp extraction based on extracted sourcetype because timestamp extraction precedes regex-based transformation in the index-time parsing sequence. Your best bet here is to configure the timestamp extraction based on the host IP address for the devices. Specifically, replace [SLB_AlteonOS] with [host::<device_ip>], and likewise for Juniper_VPN.

View solution in original post

0 Karma

Splunk Employee
Splunk Employee

You can't configure timestamp extraction based on extracted sourcetype because timestamp extraction precedes regex-based transformation in the index-time parsing sequence. Your best bet here is to configure the timestamp extraction based on the host IP address for the devices. Specifically, replace [SLB_AlteonOS] with [host::<device_ip>], and likewise for Juniper_VPN.

View solution in original post

0 Karma

Splunk Employee
Splunk Employee

The flow of index time processing is configured in $SPLUNK_HOME/etc/modules/parsing/config.xml (but don't be tempted to change it as bad things could happen).

The general order is:

  1. UTF8 decoding
  2. Line breaking
  3. Header processing (parsing intermediate configuration directives).
  4. Date parsing and line merging.
  5. Regex replacement.
  6. Addition of punct::.
0 Karma

Path Finder

Thanks. Is there a diagram or a list that describes the sequence of index-time in general?

0 Karma
State of Splunk Careers

Access the Splunk Careers Report to see real data that shows how Splunk mastery increases your value and job satisfaction.

Find out what your skills are worth!