Splunk Search

How to filter multiple regex condition and create column on basis of condition?

anirban_nag
Explorer

I have a big table of many fields. From there I want to mark result. I have a lookup conditions, if those conditions get matched then only mark them this is the logic. Example -

Say I have this data in tabular format -

+--------------+------------+--------------------+------------+--------------------------+-----------+------------+----------------------+
| IP           |  Location  |  Address           |  UserName  |  Email                   |  Pincode  |  Phn       |  Message             |
+--------------+------------+--------------------+------------+--------------------------+-----------+------------+----------------------+
| 22.33.4.5    |  UK        |  22 Baker Street   |  SHolmes   |  holmes1@elementary.com  | 450001    | 8981399334 |  Dear watson         |
+--------------+------------+--------------------+------------+--------------------------+-----------+------------+----------------------+
| 22.45.12.44  |  US        |  45 R Street       |  JimMo2    |  moes2@gmail.com         | 340003    | 7533342190 |  Hello there         |
+--------------+------------+--------------------+------------+--------------------------+-----------+------------+----------------------+
| 12.3.54.5    | UK         | 34 El Road         | Jesr       | gh.row@splunk.com        | 443434    | 2332435543 | Destorying the world |
+--------------+------------+--------------------+------------+--------------------------+-----------+------------+----------------------+
| 34.54.3.54   | RUS        | 23 RR Street       | SuperMol   | mol23@splunk.com         | 230002    | 4567575743 | Hey there            |
+--------------+------------+--------------------+------------+--------------------------+-----------+------------+----------------------+
| 214.53.3.53  | YUG        | 45 Morisson Street | Mrxtz      | mol23@splunk.com         | 2300043   | 6767556743 | Marry her            |
+--------------+------------+--------------------+------------+--------------------------+-----------+------------+----------------------+

Now I have this lookup -

+-----------+------------+------------+-----------+--------------+
| Address   |  UserName  |  Email     |  Message  |  Type        |
+-----------+------------+------------+-----------+--------------+
| *Street*  |  *         |  *gmail*   |  *        |  StreetGmail |
+-----------+------------+------------+-----------+--------------+
| *         |  *         |  *splunk*  |  *        |  Splunk      |
+-----------+------------+------------+-----------+--------------+
| *         |  *         |  *         | *Destroy* |  Banned      |
+-----------+------------+------------+-----------+--------------+
| *         |  *Super*   |  *splunk*  |  *        |  SplunkSuper |
+-----------+------------+------------+-----------+--------------+

So as per this lookup if a resultset matches with the condition then create a new column Type and map it to the table. So our original table becomes something like this -

+--------------+------------+--------------------+------------+--------------------------+-----------+------------+----------------------+-------------+
| IP           |  Location  |  Address           |  UserName  |  Email                   |  Pincode  |  Phn       |  Message             | Type        |
+--------------+------------+--------------------+------------+--------------------------+-----------+------------+----------------------+-------------+
| 22.33.4.5    |  UK        |  22 Baker Street   |  SHolmes   |  holmes1@elementary.com  | 450001    | 8981399334 |  Dear watson         | Other       |
+--------------+------------+--------------------+------------+--------------------------+-----------+------------+----------------------+-------------+
| 22.45.12.44  |  US        |  45 R Street       |  JimMo2    |  moes2@gmail.com         | 340003    | 7533342190 |  Hello there         | StreetGmail |
+--------------+------------+--------------------+------------+--------------------------+-----------+------------+----------------------+-------------+
| 12.3.54.5    | UK         | 34 El Road         | Jesr       | gh.row@splunk.com        | 443434    | 2332435543 | Destorying the world | Banned      |
+--------------+------------+--------------------+------------+--------------------------+-----------+------------+----------------------+-------------+
| 34.54.3.54   | RUS        | 23 RR Street       | SuperMol   | mol23@splunk.com         | 230002    | 4567575743 | Hey there            | SplunkSuper |
+--------------+------------+--------------------+------------+--------------------------+-----------+------------+----------------------+-------------+
| 214.53.3.53  | YUG        | 45 Morisson Street | Mrxtz      | mol23@splunk.com         | 2300043   | 6767556743 | Marry her            | Other       |
+--------------+------------+--------------------+------------+--------------------------+-----------+------------+----------------------+-------------+

Which means if the condition doesn't match mark it as Other. I think the output is pretty self-explanatory. But if someone need explanation I can give it too.

I am not sure what to use, I was checking with inputlookup but I am not sure that's the correct thing to use.

0 Karma
1 Solution

elliotproebstel
Champion

First step will be to take the data in your lookup and make sure you've defined it as a wildcard lookup. You can do that by creating a Lookup in the Splunk Web UI, specifying the csv as the Lookup file and then putting this into the field Match type: WILDCARD(Address), WILDCARD(UserName), WILDCARD(Email), WILDCARD(Message). Let's say you called the lookup get_type, so that I can refer to it below.

Next, you'll use that lookup in your search. You'll do that like this:

your base search that gives you the first table above
| lookup get_type Address, UserName, Email, Message OUTPUT Type

Then you'll fill in the Type field for events that had no matches:

| fillnull value="Other" Type

View solution in original post

elliotproebstel
Champion

First step will be to take the data in your lookup and make sure you've defined it as a wildcard lookup. You can do that by creating a Lookup in the Splunk Web UI, specifying the csv as the Lookup file and then putting this into the field Match type: WILDCARD(Address), WILDCARD(UserName), WILDCARD(Email), WILDCARD(Message). Let's say you called the lookup get_type, so that I can refer to it below.

Next, you'll use that lookup in your search. You'll do that like this:

your base search that gives you the first table above
| lookup get_type Address, UserName, Email, Message OUTPUT Type

Then you'll fill in the Type field for events that had no matches:

| fillnull value="Other" Type

anirban_nag
Explorer

This works fine but the problem arises when there is a case where Address is blank. Then the OUTPUT logic doesn't help. It giving output as Other. Can you help me here?

0 Karma
Get Updates on the Splunk Community!

Earn a $35 Gift Card for Answering our Splunk Admins & App Developer Survey

Survey for Splunk Admins and App Developers is open now! | Earn a $35 gift card!      Hello there,  Splunk ...

Continuing Innovation & New Integrations Unlock Full Stack Observability For Your ...

You’ve probably heard the latest about AppDynamics joining the Splunk Observability portfolio, deepening our ...

Monitoring Amazon Elastic Kubernetes Service (EKS)

As we’ve seen, integrating Kubernetes environments with Splunk Observability Cloud is a quick and easy way to ...