Getting Data In
Highlighted

How do I edit my props.conf and transforms.conf to filter select events from one source file from getting indexed?

New Member

We have a vanilla install, just one stand alone Splunk Server. I am wanting to filter select events from one source file. Not sure how to do it.

I have attempted to research the solution, but nothing so far has worked as expected. Maybe my expectations are not what they should be.

Here is my props.conf:

[source::"\\\\Alvionix03\\d\\InCharge\\SAM\\smarts\\local\\logs\\TRAPS-Proview.log"]
TRANSFORMS-null= discards

Here is my transforms.conf

[discards]
REGEX = Discard:\s+'YES'
DEST_KEY = queue
FORMAT = nullQueue

My expectations are that all the records which are marked as discarded in our log will not be indexed.
Example of one record of my data:

======================== Trap attributes =========================
Timestamp:           'October 27, 2015 10:54:16 AM CDT'
Agent:               '10.10.54.82'
Enterprise OID:      '.1.3.6.1.4.1.14760'
Generic Type:        '6'
Specific Type:       '1'
Varbinds:            [oid]->[varbind]
                     '.1.3.6.1.4.1.14760.2.1.2.1' --> 'A362-2250'
                     '.1.3.6.1.4.1.14760.2.1.2.2' --> '20151027095414'
                     '.1.3.6.1.4.1.14760.2.1.2.11' --> 'WFS_SYSE_DEVICE_STATUS: PhysicalName=CIM_CCDMWorkstationName=A362-2250 State=WFS_STAT_DEVONLINE (CIM_CCDM)'
                     '.1.3.6.1.4.1.14760.2.1.2.12' --> 'Device Cashin CCDM Module online'
                     '.1.3.6.1.4.1.14760.2.1.2.15' --> 'Cash/Cheque In'
=================== ICS_Notification attributes ==================
ClassName:           'Proview'
InstanceName:        'A362-2250'
EventName:           'ATM - A362-2250 - Device Cashin CCDM Module online'
Severity:            '5'
EventText:           'Proview/ATM Event: A362-2250 20151027095414 WFS_SYSE_DEVICE_STATUS: PhysicalName=CIM_CCDMWorkstationName=A362-2250 State=WFS_STAT_DEVONLINE (CIM_CCDM) Device Cashin CCDM Module online'
Category:            'SNMPTrap'
**Discard:             'YES'**
ForceOcc:            'A362-2250'
SuppressAgentOcc:    ''
UpdateUD:            ''
Expiration:          '600'
State:               'NOTIFY'
InMaintenance:       'FALSE'
ClearOnAcknowledge:  'TRUE'
TrapSource:          'Trap Processor'
EventType:           'MOMENTARY'
ASL:                 'proview.asl'
ElementClassName:    'Host'
ElementInstanceName: '10.10.54.82'
SysNameOrAddr:       'A362-2250'
UnknownAgent:        'CREATE'
LogFile:             'TRAPS-Proview.log'
UserDefined1:        '10.10.54.82'
UserDefined2:        ''
UserDefined3:        ''
UserDefined4:        ''
UserDefined5:        ''
UserDefined6:        ''
UserDefined7:        'Device Cashin CCDM Module online'
UserDefined8:        'Proview ATM Trap 1 from 10.10.54.82/10.10.54.82
MIB Module:     
wnProviewDeviceId:  A362-2250
wnProviewOriginalTime:  20151027095414
wnProviewServerTimed:   WFS_SYSE_DEVICE_STATUS: PhysicalName=CIM_CCDMWorkstationName=A362-2250 State=WFS_STAT_DEVONLINE (CIM_CCDM)
wnProviewEventType: Device Cashin CCDM Module online
wnProviewEventNumber:   Cash/Cheque In
wnProviewOriginalEventNumber:   
wnProviewDeviceState:   
wnProviewSetStateChange:    
wnProviewUnsetStateChange:  
wnProviewEventMask: 
wnProviewOriginalEventText: 
wnProviewEventText: 
wnProviewSetBitMask:    
wnProviewUnsetBitMask:  
wnProviewComponentName: 
wnProviewComponentState:    
wnProviewTransportAddress:  '
UserDefined9:        ''
UserDefined10:       ''
UserDefined11:        ''
UserDefined12:        ''
UserDefined13:        ''
UserDefined14:        ''
UserDefined15:        ''
UserDefined16:        ''
UserDefined17:        ''
UserDefined18:        ''
UserDefined19:        ''
UserDefined20:       ''
==================================================================

It is just not working at this time. I am still seeing the Discarded records indexed in Splunk.

Any assistance you can provide will be appreciated.

bfnpmsz

0 Karma
Highlighted

Re: How do I edit my props.conf and transforms.conf to filter select events from one source file from getting indexed?

SplunkTrust
SplunkTrust

Try something like this for your props.conf entry

[source::...\\d\\InCharge\\SAM\\smarts\\local\\logs\\TRAPS-Proview.log]
 TRANSFORMS-null= discards
0 Karma
Highlighted

Re: How do I edit my props.conf and transforms.conf to filter select events from one source file from getting indexed?

New Member

I removed the quotes as you suggested, seemed logical, no luck though.

[source::\\\\Alvionix03\\d\\InCharge\\SAM\\smarts\\local\\logs\\TRAPS-Proview.log]
TRANSFORMS-null= discards

Still the events are getting indexed.

Any other ideas?

bfnpmsz

0 Karma
Highlighted

Re: How do I edit my props.conf and transforms.conf to filter select events from one source file from getting indexed?

SplunkTrust
SplunkTrust

Can you try with exact stanza as mine (you seem to shared directory and I would suggest to try option without that)?

0 Karma
Highlighted

Re: How do I edit my props.conf and transforms.conf to filter select events from one source file from getting indexed?

New Member
[source::...\\d\\InCharge\\SAM\\smarts\\local\\logs\\TRAPS-Proview.log]
TRANSFORMS-null= discards

Still no love.... All records are getting indexed. The discards are still there.

bfnpmsz

0 Karma
Highlighted

Re: How do I edit my props.conf and transforms.conf to filter select events from one source file from getting indexed?

SplunkTrust
SplunkTrust

Just to confirm, you're restarting Splunk after the change?

0 Karma
Highlighted

Re: How do I edit my props.conf and transforms.conf to filter select events from one source file from getting indexed?

New Member

Yes, a restart after each config change.

0 Karma
Highlighted

Re: How do I edit my props.conf and transforms.conf to filter select events from one source file from getting indexed?

Esteemed Legend

You need to deploy these files all of your Indexers (or if using them, Heavy Forwarders) and then restart all splunk instances there. When verifying function, only check NEW events, events indexed previous to the restart will not be effected.

0 Karma
Highlighted

Re: How do I edit my props.conf and transforms.conf to filter select events from one source file from getting indexed?

New Member

Woodcock,

Yeah, I wish I have not tried that already. I only have the one server with Splunk installed and so therefore one indexer. Each time I have restarted Splunk and the Discarded records are still indexed. The old records are not affected because they have been indexed before this change.

I am not sure where I am going wrong, but something is amiss.

Thanks for your comment and help.
bfnpmsz

0 Karma
Highlighted

Re: How do I edit my props.conf and transforms.conf to filter select events from one source file from getting indexed?

Esteemed Legend

The OS is windows, right?

0 Karma