How do I search to exclude logs with extensions?

[sravani27]

Hi,
In my data, I have API calls with several extensions like (.html, .com, .php and many more). I am trying to exclude the logs that have these extensions. I tried the below.

index=abc  NOT (api_call=".html." OR api_call=".php")

But, I don't want to use NOT since there are many extensions that will come in the future.

Can anyone help?

[horsefez]

Hi @sravani27,

As you don't want to use the NOT boolean expression to filter out the extensions, I have a suggestion how you can achieve your goal.

I would go for a lookup table that looks like this:
file_extensions.csv

extension,exclude
html,1
com,1
php,1

Upload that lookup to your system and add a lookup definition so you can use it in your search.

Then go ahead and write a search like this:

index=abc api_call=*
| rex field=api_call "(?<extracted_file_extension>[^\.]+?$)"
| inputlookup file_extensions extension AS extracted_file_extension OUTPUT exclude
| where isnull(exclude)

With that query you should be able to exclude all the unwanted extension types. And if you need to exclude more extensions in the future just add them to your csv.

Hope this helps 😃

[horsefez]

@sravani27,

if you found my solution helpful and it worked for you please accept the answer.

Thanks!

[ddrillic]

You can extract the extension into a field and base your query on this field.

[adonio]

if the field api_call has only extensions, yow can do: index=abc NOT api_call=*
if the field api_call has the file and extension you do: index=abc NOT api_call=.*

let us know if any of the above works for you as there are other ways to accomplish.