Getting Data In

line breaking...

Champion

Hi,

I'm stumped. I've been playing with the linebreaking trying to get the format properly, and it won't work. The format is below. I want each "Trap:" to begin a new event, down to the next "Trap:" Any suggestions?

Trap: 23708419
Wed Feb 8 02:01:11 2012
Src IP: 10.216.0.26
Agent IP: 10.216.0.26
Trap Type: Vendor Specific
Specific Type: 1
Enterprise: 1.3.6.1.4.1.9.9.41.2
Object:1.3.6.1.4.1.9.9.41.1.2.3.1.2.61290766 Value:PIM
Object:1.3.6.1.4.1.9.9.41.1.2.3.1.3.61290766 Value:5
Object:1.3.6.1.4.1.9.9.41.1.2.3.1.4.61290766 Value:INVALID_SRC_REG
Object:1.3.6.1.4.1.9.9.41.1.2.3.1.5.61290766 Value:Received Register from XX.XX.XX.XX for (XX.XX.XX.XX, XX.XX.XX.XXX), not willing to be RP
Object:1.3.6.1.4.1.9.9.41.1.2.3.1.6.61290766 Value:467d 06:45:53

Trap: 23708420
Wed Feb 8 02:01:11 2012
Src IP: 1.2.3.4
Agent IP: 1.2.3.4
Trap Type: Authentication Failure
Specific Type: 0
Enterprise: 1.3.6.1.6.3.1.1.5
Object:1.3.6.1.4.1.9.2.1.5.0 Value:1.2.3.4
Object:1.3.6.1.4.1.9.9.412.1.1.1.0 Value:1
Object:1.3.6.1.4.1.9.9.412.1.1.2.0 Value:1.2.3.4

Trap: 23708421
Wed Feb 8 02:01:11 2012
Src IP: 1.2.3.4
Agent IP: 1.2.3.4
Trap Type: Authentication Failure
Specific Type: 0
Enterprise: 1.3.6.1.6.3.1.1.5
Object:1.3.6.1.4.1.9.2.1.5.0 Value:1.2.3.4
Object:1.3.6.1.4.1.9.9.412.1.1.1.0 Value:1
Object:1.3.6.1.4.1.9.9.412.1.1.2.0 Value:1.2.3.4

Tags (1)
0 Karma
1 Solution

Builder

You will need to configure props.conf like as bellow.

[your_sourcetype]
SHOULD_LINEMERGE = True
BREAK_ONLY_BEFORE = Trap:

You can also refer to following manual.

http://docs.splunk.com/Documentation/Splunk/latest/Data/Indexmulti-lineevents

View solution in original post

Builder

You will need to configure props.conf like as bellow.

[your_sourcetype]
SHOULD_LINEMERGE = True
BREAK_ONLY_BEFORE = Trap:

You can also refer to following manual.

http://docs.splunk.com/Documentation/Splunk/latest/Data/Indexmulti-lineevents

View solution in original post

Motivator

A good technique for this is to do go to Settings->Data Inputs->Add New (Files & Directories) on your indexer with a sample log file in the temp directory, say. Select Preview Data Before Indexing and then Browse for the file. Once you've got that, click Continue.

In the new screen called Data Preview, you get a pop-up asking for you to select a sourcetype from the list of known ones, or to create a new sourcetype. If you use an existing sourcetype, Splunk will use the props.conf stanza associated with that sourcetype on the indexer (if there is one), and pre-populate the settings in the Advanced Mode tab with them. Once you've done this (selected which option on sourcetype), you can see how Splunk is parsing the logs. Typically, if they are easy to parse then date and time (timestamp) in the logs will be highlighted in green. If not, you'll see a warning icon on the lines it can't figure out.

This is where this is a nice tool. You can go to the Advanced Mode (props.conf) tab and in the Additional Settings (override) block enter in your various props.conf settings you'd like to try, then Apply them. To this point, none of the things you have done affect the configuration of the indexer in any way, and you get to see the effects of the different things you try there.

0 Karma

Champion

No, you understand it, but that's not what's happening. See below from a search...

Wed Feb 8 20:53:27 2012
Src IP: 1..2.3.4
Agent IP: 1.2.3.4
Trap Type: Vendor Specific
Specific Type: 0
Enterprise: 1.3.6.1.4.1.2620.1.5.6
Object:1.3.6.1.4.1.2620.1.5.6.0 Value:standby
Object:1.1.1.0 Value:Cluster State
Trap: 24678117 <---- this should be the start of the next trap.

0 Karma

Builder

Did you restart splunk and reflect configuration of props.conf and clean indexed data? The configuration will reflect for new index data, not past indexed data.

0 Karma

Builder

Are you using forwarder? Do you know where you should put props.conf in your deployment?

If you are using light weight forwarder or universal forwarder, you need to put the props.conf on index server.
If you are using other forwarder type(HF or regular forwarder), you will need to put the props.conf on forwarder, not index server. Please confirm if you put props.conf on appropriate location. You will also need to restart splunk to reflect the configuration.

0 Karma

Builder

can you vote for me?

0 Karma

Champion

tHANKS! I appreciate it.

0 Karma

Champion

That did it! Thanks!

0 Karma

Builder

You need to vote for me, not your self.....

0 Karma

Champion

OK. I'll try that. Thanks!

0 Karma

Champion

I'm using the universal forwarder. So, the props.conf needs to go on the index server?

0 Karma

Builder

Yes, you need to put the props.conf on the index server.

0 Karma

Champion

Yeah, restarted the forwarder, and the results are below.

Wed Feb 8 23:10:18 2012
Src IP: 1.2.3.4
Agent IP: 1.2.3.4
Trap Type: Authentication Failure
Specific Type: 0
Enterprise: 1.3.6.1.6.3.1.1.5
Object:1.3.6.1.4.1.9.2.1.5.0 Value:1.2.3.4
Trap: 24780942

0 Karma

Champion

Thanks. That almost works. It's putting the "Trap:" from the next event at the bottom of the previous event. The "Trap:" is the start of the event, and I want to include it. Any way to do that?

0 Karma

Builder

"Trap:" is the start of the event. If you break before "Trap:", you will see "Trap:" is first line of the indexed event. Please let me know if I have misunderstanding.

0 Karma
State of Splunk Careers

Access the Splunk Careers Report to see real data that shows how Splunk mastery increases your value and job satisfaction.

Find out what your skills are worth!