Hello,
I need to check the regex condition only on first 300 characters, if the regex condition available after than then it should ignore the event.
Is it possible?
Solved the issue by Using LOOKAHEAD option available in transforms.conf
https://docs.splunk.com/Documentation/Splunk/7.2.6/Admin/Transformsconf
[route_to_index]
LOOKAHEAD=300
Solved the issue by Using LOOKAHEAD option available in transforms.conf
https://docs.splunk.com/Documentation/Splunk/7.2.6/Admin/Transformsconf
[route_to_index]
LOOKAHEAD=300
Check the new comment I added to my answer as you might run into some issues with this config.
Hi vishaltaneja07011993,
I cannot test this but it should run.
my_search
| eval _raw=substr(_raw,1,300)
| rex field=_raw "your_regex"
| ...
Bye.
Giuseppe
I need to do it at index time. based on a word on first 300 charters of event we need to change the index. Like debug or error logs.
Is this something you want to do at search time, or do you want to use this to do some routing/filtering at index time?
If at search time, you could extract the first 300 characters into a field and then apply your regex to that field.
And is that 300 character really the only way to determine where the regex should look? Is there no recognizable pattern in the event behind which the regex should not apply?
Hello @FrankVI
I need to do it at index time. based on a word on first 300 charters of event we need to change the index. Like debug or error logs.
Hi @vishaltaneja07011993,
If you want to match your regex on the first 300 characters only, you can use a regex like this one and add any character you want to allow :
^[a-z]{0,300}$
This will match only on 0->300 characters.
Cheers,
David
Need to do it index time. And we need to match a specific word only
I need to do it at index time. based on a word on first 300 charters of event we need to change the index. Like debug or error logs.
Have a look here for a good example of index time extractions :
https://answers.splunk.com/answers/728933/can-you-help-me-create-the-regex-for-an-index-time.html
What exactly do you want to extract ? Is there a specific word in the first 300 characters ?
Like if there is debug word there in event but in first 300 characters only , then the event should go to debug index ,not the normal one .
Careful when using lookahead
as this might affect line breaking for the events that contain more than 300 characters. Go for something like this instead :
In props.conf
[mysourcetype]
TRANSFORMS-newindex = index2debug, OtherIndex
in transforms.conf
[index2debug]
DEST_KEY =_MetaData:Index
REGEX = ^.{0,300}debug
FORMAT = debugindex
Cheers,
David
Hello @DavidHourani
I have applied Lookahead for that particular transforms stanza only. so i dnt think it will affect the line breaking.
Also, It is not working, tried with below stanza
06/04/2019 02:55:06 AM
LogName=abccccccccccc
SourceName=fjdjhfjhjdjhjdhfhfjhjd
EventCode=hdfhdfjd
EventType=fdfdfdfd
Type=Error
ComputerName=dfdfdfdfdf.dfdfd.dfdf
dfdfd
jajhsjhdjjdhjhjsjhjujjfjhhsjhjhfjjhjjdghjjfhjjdhjjdhjfhhhhdhdhdhhhhdhhhdhhhhhhdg
jdhjdjfhjdhjdfhjdjdfjdhfjdjhfdjfh
kfkkjkfhdfkdfhkfhkfhdf
Severity: debug
Still the above regex is accepting debug log which is at more than 300 characters away.
That looks like nicely structured logs (windows logs by the look of it?). Why not simply filter for type=Debug
, instead of depending on the 300 char limit?
@FrankVl
I have given one example the format changes in different logs, i need some generalised thing which can work for all kind of logs.
Its actually 300max then the word debug so 305 ^^
And yes, if youve tested your solution and it didnt backfire then please ignore my comment :):)