Regex: Can it be possible to extract one common field if we have two sourcetypes and different sourcepath?

[shruti14]

Hi,

Can it be possible to extract one common field if we have two sourcetypes and sourcepath is also different in them , index is same.

Example : sourcetype : abc with source path : /home/mysqld/$DB_NAME/audit/audit.log

sourcetype:xyz with sourcepath : /mydata/log/$DB_NAME/audit/audit.log

I need to have DBname extracted is that possible to get it via regex if yes what it can be.

Also if not can i make soucretype as one with 2 different sourcepath /home/mysqld/$DB_NAME/audit/audit.log and /mydata/log/$DB_NAME/audit/audit.log 

and then extract DBname from it via regex?

[shruti14]

ok and if sourcetype is different then probably if i give below one that would be fine :

(sourcetype=A OR sourcetype=B)

| rex field=source "/home/mysqld/(?<DB_NAME1>[^/]+)/"
| rex field=source "/mydata/log/(?<DB_NAME2>[^/]+)/"
```Combine the two DB_NAME fields```
| eval DB_NAME = coalesce(DB_NAME1, DB_NAME2)
| fields - DB_NAME1, DB_NAME2

[richgalloway]

Yes, that should be fine.

[shruti14]

"record":{"name”:"abc","record":"3055975400_2022-05-28T18:13:38","timestamp":"2022-08-24T11:33:47 UTC","connection_id":"3316408","status":0,"user":"exporter","user":"exporter","os_login":"","proxy_user":"","host”:”abc.com","ip”:"xxxxx","db":""}}

this raw text

[richgalloway]

In that case, you'll probably need two rex commands.  You may be able to craft a regex that includes both path styles, but I prefer the simplicity of separate commands.

| rex field=source "/home/mysqld/(?<DB_NAME1>[^/]+)/"
| rex field=source "/mydata/log/(?<DB_NAME2>[^/]+)/"
```Combine the two DB_NAME fields```
| eval DB_NAME = coalesce(DB_NAME1, DB_NAME2)
| fields - DB_NAME1, DB_NAME2

[shruti14]

yeah but i have 2 soucre giving databases and they both have common value but comes at different path alsi in raw data i dont have database name.

[richgalloway]

By default, the rex command searches _raw, which encompasses all fields.  You could get the DB name using something like this.

| rex "/mydata/log/(?<DB_NAME>[^\/]+)/"

[shruti14]

Also i don't want to harcode it.