topic Re: line breaking... in Getting Data In

line breaking...

a212830 — Mon, 28 Sep 2020 11:22:06 GMT

Hi,

I'm stumped. I've been playing with the linebreaking trying to get the format properly, and it won't work. The format is below. I want each "Trap:" to begin a new event, down to the next "Trap:" Any suggestions?

Trap: 23708419
Wed Feb 8 02:01:11 2012
Src IP: 10.216.0.26
Agent IP: 10.216.0.26
Trap Type: Vendor Specific
Specific Type: 1
Enterprise: 1.3.6.1.4.1.9.9.41.2
Object:1.3.6.1.4.1.9.9.41.1.2.3.1.2.61290766 Value:PIM
Object:1.3.6.1.4.1.9.9.41.1.2.3.1.3.61290766 Value:5
Object:1.3.6.1.4.1.9.9.41.1.2.3.1.4.61290766 Value:INVALID_SRC_REG
Object:1.3.6.1.4.1.9.9.41.1.2.3.1.5.61290766 Value:Received Register from XX.XX.XX.XX for (XX.XX.XX.XX, XX.XX.XX.XXX), not willing to be RP
Object:1.3.6.1.4.1.9.9.41.1.2.3.1.6.61290766 Value:467d 06:45:53

Trap: 23708420
Wed Feb 8 02:01:11 2012
Src IP: 1.2.3.4
Agent IP: 1.2.3.4
Trap Type: Authentication Failure
Specific Type: 0
Enterprise: 1.3.6.1.6.3.1.1.5
Object:1.3.6.1.4.1.9.2.1.5.0 Value:1.2.3.4
Object:1.3.6.1.4.1.9.9.412.1.1.1.0 Value:1
Object:1.3.6.1.4.1.9.9.412.1.1.2.0 Value:1.2.3.4

Trap: 23708421
Wed Feb 8 02:01:11 2012
Src IP: 1.2.3.4
Agent IP: 1.2.3.4
Trap Type: Authentication Failure
Specific Type: 0
Enterprise: 1.3.6.1.6.3.1.1.5
Object:1.3.6.1.4.1.9.2.1.5.0 Value:1.2.3.4
Object:1.3.6.1.4.1.9.9.412.1.1.1.0 Value:1
Object:1.3.6.1.4.1.9.9.412.1.1.2.0 Value:1.2.3.4

Re: line breaking...

Takajian — Mon, 28 Sep 2020 11:22:09 GMT

You will need to configure props.conf like as bellow.

[your_sourcetype]
SHOULD_LINEMERGE = True
BREAK_ONLY_BEFORE = Trap:

You can also refer to following manual.

http://docs.splunk.com/Documentation/Splunk/latest/Data/Indexmulti-lineevents

Re: line breaking...

a212830 — Thu, 09 Feb 2012 01:35:42 GMT

Thanks. That almost works. It's putting the "Trap:" from the next event at the bottom of the previous event. The "Trap:" is the start of the event, and I want to include it. Any way to do that?

Re: line breaking...

Takajian — Thu, 09 Feb 2012 01:40:56 GMT

"Trap:" is the start of the event. If you break before "Trap:", you will see "Trap:" is first line of the indexed event. Please let me know if I have misunderstanding.

Re: line breaking...

a212830 — Thu, 09 Feb 2012 04:12:42 GMT

Yeah, restarted the forwarder, and the results are below.

Wed Feb 8 23:10:18 2012
Src IP: 1.2.3.4
Agent IP: 1.2.3.4
Trap Type: Authentication Failure
Specific Type: 0
Enterprise: 1.3.6.1.6.3.1.1.5
Object:1.3.6.1.4.1.9.2.1.5.0 Value:1.2.3.4
Trap: 24780942

Re: line breaking...

a212830 — Thu, 09 Feb 2012 04:18:24 GMT

I'm using the universal forwarder. So, the props.conf needs to go on the index server?

Re: line breaking...

Takajian — Thu, 09 Feb 2012 04:22:14 GMT

Yes, you need to put the props.conf on the index server.

Re: line breaking...

a212830 — Thu, 09 Feb 2012 04:22:50 GMT

OK. I'll try that. Thanks!

Re: line breaking...

a212830 — Thu, 09 Feb 2012 04:34:10 GMT

That did it! Thanks!

Re: line breaking...

Takajian — Thu, 09 Feb 2012 04:36:51 GMT

You need to vote for me, not your self.....

Re: line breaking...

a212830 — Tue, 22 May 2012 21:28:09 GMT

tHANKS! I appreciate it.

Re: line breaking...

Takajian — Tue, 22 May 2012 23:23:03 GMT

can you vote for me?

Re: line breaking...

Takajian — Mon, 06 Jul 2015 21:25:16 GMT

Are you using forwarder? Do you know where you should put props.conf in your deployment?

If you are using light weight forwarder or universal forwarder, you need to put the props.conf on index server.
If you are using other forwarder type(HF or regular forwarder), you will need to put the props.conf on forwarder, not index server. Please confirm if you put props.conf on appropriate location. You will also need to restart splunk to reflect the configuration.

Re: line breaking...

a212830 — Mon, 06 Jul 2015 21:25:17 GMT

No, you understand it, but that's not what's happening. See below from a search...

Wed Feb 8 20:53:27 2012
Src IP: 1..2.3.4
Agent IP: 1.2.3.4
Trap Type: Vendor Specific
Specific Type: 0
Enterprise: 1.3.6.1.4.1.2620.1.5.6
Object:1.3.6.1.4.1.2620.1.5.6.0 Value:standby
Object:1.1.1.0 Value:Cluster State
Trap: 24678117 <---- this should be the start of the next trap.

Re: line breaking...

Takajian — Mon, 06 Jul 2015 21:25:18 GMT

Did you restart splunk and reflect configuration of props.conf and clean indexed data? The configuration will reflect for new index data, not past indexed data.

Re: line breaking...

wrangler2x — Mon, 06 Jul 2015 21:55:23 GMT

A good technique for this is to do go to Settings->Data Inputs->Add New (Files & Directories) on your indexer with a sample log file in the temp directory, say. Select Preview Data Before Indexing and then Browse for the file. Once you've got that, click Continue.

In the new screen called Data Preview, you get a pop-up asking for you to select a sourcetype from the list of known ones, or to create a new sourcetype. If you use an existing sourcetype, Splunk will use the props.conf stanza associated with that sourcetype on the indexer (if there is one), and pre-populate the settings in the Advanced Mode tab with them. Once you've done this (selected which option on sourcetype), you can see how Splunk is parsing the logs. Typically, if they are easy to parse then date and time (timestamp) in the logs will be highlighted in green. If not, you'll see a warning icon on the lines it can't figure out.

This is where this is a nice tool. You can go to the Advanced Mode (props.conf) tab and in the Additional Settings (override) block enter in your various props.conf settings you'd like to try, then Apply them. To this point, none of the things you have done affect the configuration of the indexer in any way, and you get to see the effects of the different things you try there.