Getting Data In

Regex: Can it be possible to extract one common field if we have two sourcetypes and different sourcepath?

shruti14
Explorer

Hi,

Can it be possible to extract one common field if we have two sourcetypes and sourcepath is also different in them , index is same.

Example : sourcetype : abc with source path : /home/mysqld/$DB_NAME/audit/audit.log

sourcetype:xyz with sourcepath : /mydata/log/$DB_NAME/audit/audit.log

I need to have DBname extracted is that possible to get it via regex if yes what it can be.

Also if not can i make soucretype as one with 2 different sourcepath /home/mysqld/$DB_NAME/audit/audit.log and /mydata/log/$DB_NAME/audit/audit.log 

and then extract DBname from it via regex?

 

Labels (2)
Tags (2)
0 Karma

shruti14
Explorer

ok and if sourcetype is different then probably if i give below one that would be fine :

(sourcetype=A OR sourcetype=B)

 

| rex field=source "/home/mysqld/(?<DB_NAME1>[^/]+)/"
| rex field=source "/mydata/log/(?<DB_NAME2>[^/]+)/"
```Combine the two DB_NAME fields```
| eval DB_NAME = coalesce(DB_NAME1, DB_NAME2)
| fields - DB_NAME1, DB_NAME2
0 Karma

richgalloway
SplunkTrust
SplunkTrust

Yes, that should be fine.

---
If this reply helps you, Karma would be appreciated.
0 Karma

shruti14
Explorer

"record":{"name:"abc","record":"3055975400_2022-05-28T18:13:38","timestamp":"2022-08-24T11:33:47 UTC","connection_id":"3316408","status":0,"user":"exporter","user":"exporter","os_login":"","proxy_user":"","host:abc.com","ip:"xxxxx","db":""}}

this raw text

0 Karma

richgalloway
SplunkTrust
SplunkTrust

In that case, you'll probably need two rex commands.  You may be able to craft a regex that includes both path styles, but I prefer the simplicity of separate commands.

| rex field=source "/home/mysqld/(?<DB_NAME1>[^/]+)/"
| rex field=source "/mydata/log/(?<DB_NAME2>[^/]+)/"
```Combine the two DB_NAME fields```
| eval DB_NAME = coalesce(DB_NAME1, DB_NAME2)
| fields - DB_NAME1, DB_NAME2

 

---
If this reply helps you, Karma would be appreciated.
0 Karma

shruti14
Explorer

yeah but i have 2 soucre giving databases and they both have common value but comes at different path alsi in raw data i dont have database name.

0 Karma

richgalloway
SplunkTrust
SplunkTrust

By default, the rex command searches _raw, which encompasses all fields.  You could get the DB name using something like this.

| rex "/mydata/log/(?<DB_NAME>[^\/]+)/"
---
If this reply helps you, Karma would be appreciated.
0 Karma

shruti14
Explorer

Also i don't want to harcode it.

0 Karma
Get Updates on the Splunk Community!

Technical Workshop Series: Splunk Data Management and SPL2 | Register here!

Hey, Splunk Community! Ready to take your data management skills to the next level? Join us for a 3-part ...

Spotting Financial Fraud in the Haystack: A Guide to Behavioral Analytics with Splunk

In today's digital financial ecosystem, security teams face an unprecedented challenge. The sheer volume of ...

Solve Problems Faster with New, Smarter AI and Integrations in Splunk Observability

Solve Problems Faster with New, Smarter AI and Integrations in Splunk Observability As businesses scale ...