I have filter applied in transforms.conf as follows [send_to_heavy_forwarder] CAN_OPTIMIZE = True CLEAN_KEYS = True DEFAULT_VALUE = DEST_KEY = _TCP_ROUTING FORMAT = heavy_forwarder KEEP_EMPTY_VALS = False LOOKAHEAD = 4096 MATCH_LIMIT = 100000 MV_ADD = False RECURSION_LIMIT = 1000 REGEX = (logtype::ABC.*id::IDB-28123.*username::((?!-TEST).)*$) SOURCE_KEY = _meta WRITE_META = False All I'm trying here is to filter sending logs If the following conditions satisfies logtype=ABC, id=IDB-28123 and username value doesn't end with TEST which is not working but it is working if I removed the username part in the regex. It's not working before due to the negative look back I applies. If so, how can I filter those test user logs? Any help would be great. Check below for more details :- Logs in Indexer :- pppppppppppp logtype::ABC id::IDB-28123 pppppppp username::ppppppp pppppppppppp logtype::ABC id::IDB-28123 pppppppp username::qqq pppppppppppp logtype::ABC id::IDB-28123 pppppppp username::rrTEST Regex should skip sending the below logs :- pppppppppppp logtype::ABC id::IDB-28123 pppppppp username::rrTEST Regex should send the below logs :- pppppppppppp logtype::ABC id::IDB-28123 pppppppp username::ppppppp pppppppppppp logtype::ABC id::IDB-28123 pppppppp username::qqq ... View more

The following are my transforms.conf and props.conf in my cluster master which are sending all the logs for the below search logtype=SAT (id="ABC-1" OR id="ABC-2") transforms.conf [send_to_heavyforwarder] SOURCE_KEY = _meta REGEX = (logtype::SAT.*id::(ABC-1|ABC-2)) DEST_KEY = _TCP_ROUTING FORMAT = heavyforwarder_output props.conf [default] TRANSFORMS-heavyforwarder= send_to_heavyforwarder Now, I want to filter the events when the below search condition met logtype=SAT id="ABC-2" username="anything that ends with -TEST" In order to filter the events that match the above condition how to modify my REGEX in transforms.conf. I think I need to use to something like negative look-back but not sure what could be my new syntax that filters the events when username ends with "-TEST" Any help would be great. ... View more

The following are my transforms.conf and props.conf in my cluster master transforms.conf [send_to_heavyforwarder] SOURCE_KEY = _meta REGEX = (logtype::SAT.*id::(ABC-1|ABC-2)) DEST_KEY = _TCP_ROUTING FORMAT = heavyforwarder_output props.conf [default] TRANSFORMS-heavyforwarder= send_to_heavyforwarder Which is used to send the logs to a heavy forwarder for only the fields in regex condition specified in transforms.conf matched which is below REGEX = (logtype::SAT.*id::(ABC-1|ABC-2)) Now I need to send one more id (whose field value is ABC-3) to the heavyforwarder but instead of sending the whole logs to heavyforwarder I need to exclude sending logs to heavyforwarder when an username field ends with string "TEST". For that I have modified my regex in transforms.conf as below REGEX = (logtype::SAT.*id::(ABC-1|ABC-2)) | (logtype::SAT.*id::ABC-3.*username!::[a-zA-Z]{2,3}-+.*?-TEST) Could anyone verify the above regex and confirm me if I used the right condition to satify the requirement. If not, what could be the proper REGEX syntax to be applied for my transforms.conf. Any help could be great. Note :- username is also a meta data field. ... View more

The following is transforms.conf in my search head [a_b] SOURCE_KEY = _meta REGEX = (logtype::A.*(id::(123|456)|(id::789.*username!::[a-zA-Z]{2,3}-+.*?-ZLX)) DEST_KEY = _ghi FORMAT = KLMN Now how to write my props.conf in order to test the REGEX in the above transforms.conf works. Especially I would like to see if the id=789 and username not equall to the string that ends with -ZLX? ... View more

I have a field which contains 2 values for every event as shown below: Field Name :- Username Example Values :- A,B Now from the above example I have defined 2 extra fields first_user and second_user first_user=A (1st field value from Username) second_user=B (2nd field value from Username) Both the above 2 fields work well as long as the Username field has two values but when the Username has only 1 value like the first field(A) is there and the second field(B) is null. I want to display the Null or empty for second_user field but instead right now it just picking up the A value for second_user field. In order to resolve this, I'm trying to work on a Regex on props.conf by using some if condition and a mvindex command to give the logic something like if the B is null. I want to display null. Any help on providing the format for regex would be great. ... View more

I have an eval condition as below in my search: | eval body= username. " user attempted to delete " . activity_count . " logs by performing the below activities " . mvjoin(activity,", ") . " and removed the logs on host " . src Which gives the result as follows: X user attempted to delete 2 logs by performing the below activities rm /abc/cg.log, rm -rf /def/gc.log and removed the logs on host Y Now, how can I add a line break as shown below: body X user attempted to delete 2 logs by performing the below activities rm /abc/cg.log rm -rf /def/gc.log and removed the logs on host Y Please let me know how to modify my eval condition to get the required result as shown above. ... View more

I have the following stanza on the transforms.conf which actually splits commands separated by characters like |, &, ; etc. and extracts arguments in a separate field [split_and_extract_unix_commands] SOURCE_KEY = activity REGEX =((?<prefix>[^|&#; ]*\/)?(?<command>[^|&#; ]+)( *(?<arguments>[^&|#;]*(-exec)|[^&|#;]*))?) MV_ADD = TRUE And I have the field extraction for "activity" in props.conf as follows [syslog] EXTRACT-syslog_username_command_sudo = (sudo|dzdo).* (?<username>[^\s]+) :.*USER=(?<target_account>\S+).*COMMAND=(?<activity>.+) This actually works well and splits the values of activity field into command and arguments Now how can I implement the same on a search query and create the fields "command" and "arguments" instead of creating those field extractions on transforms.conf The following the query I came across so far and not sure what's the stanza in rex for creating the fields "command" and "arguments" which actually splits the "command" field by using the transforms.conf stanza. sourcetype=syslog | rex field=_raw "(sudo|dzdo).* (?<username>[^\s]+) :.*USER=(?<target_account>\S+).*COMMAND=(?<activity>.+)" Any help would be great. Thanks in Advance. ... View more

Base query :- sourcetype=syslog How can I or where can I find if anyone removed any log files on unix syslog server? what are the sample unix commands that used to remove any log files? Any inputs would be highly appriciated. ... View more