EDITED to add relevant info:
I'm trying to prevent indexing of entries containing certain strings (ACDB0000,ACM0033,W0032,L0041, \[DEBUG\])
This stanza worked fine when all I wanted to filter was debug entries:
#old transforms.conf
[setnull]
REGEX = \[DEBUG\])
DEST_KEY = queue
FORMAT = nullQueue
But when I add a few more string to the REGEX, failure ensues. The log events containing ACM0033 and ACDB0000 aren't getting filtered out. They are still getting indexed. This is the new transforms file:
#new transforms.conf
[setnull]
REGEX = (W0032|L0041|ACM0033|ACDB0000|\[DEBUG\])
DEST_KEY = queue
FORMAT = nullQueue
This is my props.conf for both configs:
# same props.conf
[default]
TRANSFORMS-null = setnull
CHARSET = AUTO
NO_BINARY_CHECK = 1
pulldown_type = 1
[foo-prod]
TIME_FORMAT = %b %d %H:%M:%S
NO_BINARY_CHECK = 1
pulldown_type = 1
The log entries in question look something like this. A date, a level of severity in brackets and then a string of varying length. All generated in the standard Unix syslog format
2014-05-13 22:56:20,988 [INFO] ACDB0000: ACDB_LOG - IncomingRequest. guid=AN-ON method=register idx=0 <soap:Envelope xmlns:soap="http://schemas.xmlsoap.org/soap/envelope/"><soap:Body><ns2:register
What is the reason for changing the format from nullQueue to indexQueue?
Also, what is the function of:
[changeme_setnull]
REGEX = .
DEST_KEY = queue
FORMAT = nullQueue
I would appreciate it if you would explain your reasoning to me so I could learn more.
I have edited my post to answer your question. Please let me know if you'd like more info.
[default]
TIME_FORMAT = %Y-%m-%d %H:%M:%S,%9N
TRANSFORMS-changeme = changeme_setnull, changeme_setparsing
CHARSET = AUTO
NO_BINARY_CHECK = 1
pulldown_type = 1
[changeme_setparsing]
REGEX = (W0032|L0041|ACM0033|ACDB0000|[DEBUG])
DEST_KEY = queue
FORMAT = indexQueue
[changeme_setnull]
REGEX = .
DEST_KEY = queue
FORMAT = nullQueue
What kind of log u have there ? I'm not sure but you can try this. (kinda new to splunk so bare with me)
What is the reason for changing the format from nullQueue to indexQueue?
Also, what is the function of:
[changeme_setnull]
REGEX = .
DEST_KEY = queue
FORMAT = nullQueue
I would appreciate it if you would explain your reasoning to me so I could learn more.
I tried this search query:
W0032 OR L0041 OR ACM0033 OR ACDB0000 OR \[DEBUG\]
It finds ACM0033 and ACDB0000 entries but DEBUG, W0032 and L0041 appear to be filtered properly.
Am I wrong or its a multiline log ? Did u try your regex in a search query ?
btw: I've edited the transforms.conf check that out if it works.
As Rich said, can u provide some sample ? And what was the outcome ?
they are syslog generated on linux/solaris machines
How is it not working? Filtering too much or too little? Please provide some sample log events.