topic Re: Need help with the correct regex in Splunk Search

Need help with the correct regex

mninansplunk — Wed, 13 Nov 2024 22:42:46 GMT

Hello,

I'm trying to extract fields from an event, but am not up to par on my regex, and I can't seem to get this to work. So these work in regex101, but not within the Splunk Field Extraction for some reason. Within the event there is the following:

"alias":"FL-NS-VPX-INT-1|mobileapist?vnetapis003?8777,"

I need to create 3 fields from this:

Host = FL-NS-VPX-INT-1 ServiceGroup = mobileapist Server = vnetapis003

When trying for Host with:

(?<="alias":")[^|]*

It never finds it in Splunk. Can't figure out why.

Extra credit: 🙂 Just kidding. The last field I need, I can't get either with: (?<="team","name":")[^"]*

"team","name":"Monitoring_Admin"}],

Here's the full event as well.

[36mINFO[0m[2024-11-13T13:37:23.9114215-05:00] Message body: {"actionType":"custom","customerId":"3a1f4387-b87b-4a3a-a568-cc372a86d8e4","ownerDomain":"integration","ownerId":"8b500163-8476-4b0e-9ef7-2cfdaa272adf","discardScriptResponse":true,"sendCallbackToStreamHub":false,"requestId":"18dcdb1b-14d6-4b10-ad62-3f73acaaef2a","action":"Close","productSource":"Opsgenie","customerDomain":"siteone","integrationName":"Opsgenie Edge Connector","integrationId":"8b500163-8476-4b0e-9ef7-2cfdaa272adf","customerTransitioningOrConsolidated":false,"source":{"name":"","type":"system"},"type":"oec","receivedAt":1731523037863,"ownerId":"8b500163-8476-4b0e-9ef7-2cfdaa272adf","params":{"type":"oec","alertId":"913a3db5-7e2a-44f4-a4ff-3002af480c8d-1731522737697","customerId":"3a1f4387-b87b-4a3a-a568-cc372a86d8e4","action":"Close","integrationId":"8b500163-8476-4b0e-9ef7-2cfdaa272adf","integrationName":"Opsgenie Edge Connector","integrationType":"OEC","customerDomain":"siteone","alertDetails":{"Raw":"","Results Link":"https://hostname:8000/app/search/search?q=%7Cloadjob%20scheduler__td26605__search__RMD5e461b39d4ff19795_at_1731522600_38116%20%7C%20head%204%20%7C%20tail%201&earliest=0&latest=now","SuppressClosed":"True","TeamsDescription":"True"},"alertAlias":"FL-NS-VPX-INT-1|mobileapist?vnetapis003?8777,","receivedAt":1731523037863,"customerConsolidated":false,"customerTransitioningOrConsolidated":false,"productSource":"Opsgenie","source":{"name":"","type":"system"},"alert":{"alertId":"913a3db5-7e2a-44f4-a4ff-3002af480c8d-1731522737697","id":"913a3db5-7e2a-44f4-a4ff-3002af480c8d-1731522737697","type":"alert","message":"[Splunk] Load Balancer Member Status","tags":[],"tinyId":"14585","entity":"","alias":"FL-NS-VPX-INT-1|mobileapist?vnetapis003?8777,","createdAt":1731522737697,"updatedAt":1731523038582000000,"username":"System","responders":[{"id":"f8c9079d-c7bb-4e58-ac83-359cb217a3b5","type":"team","name":"Monitoring_Admin"}],"teams":["f8c9079d-c7bb-4e58-ac83-359cb217a3b5"],"actions":[],"priority":"P3","oldPriority":"P3","source":"Splunk"},"entity":{"alertId":"913a3db5-7e2a-44f4-a4ff-3002af480c8d-1731522737697","id":"913a3db5-7e2a-44f4-a4ff-3002af480c8d-1731522737697","type":"alert","message":"[Splunk] Load Balancer Member Status","tags":[],"tinyId":"14585","entity":"","alias":"FL-NS-VPX-INT-1|mobileapist?vnetapis003?8777,","createdAt":1731522737697,"updatedAt":1731523038582000000,"username":"System","responders":[{"id":"f8c9079d-c7bb-4e58-ac83-359cb217a3b5","type":"team","name":"Monitoring_Admin"}],"teams":["f8c9079d-c7bb-4e58-ac83-359cb217a3b5"],"actions":[],"priority":"P3","oldPriority":"P3","source":"Splunk"},"mappedActionDto":{"mappedAction":"postActionToOEC","extraField":""},"ownerId":"8b500163-8476-4b0e-9ef7-2cfdaa272adf"},"integrationType":"OEC","alert":{"alertId":"913a3db5-7e2a-44f4-a4ff-3002af480c8d-1731522737697","id":"913a3db5-7e2a-44f4-a4ff-3002af480c8d-1731522737697","type":"alert","message":"[Splunk] Load Balancer Member Status","tags":[],"tinyId":"14585","entity":"","alias":"FL-NS-VPX-INT-1|mobileapist?vnetapis003?8777,","createdAt":1731522737697,"updatedAt":1731523038582000000,"username":"System","responders":[{"id":"f8c9079d-c7bb-4e58-ac83-359cb217a3b5","type":"team","name":"Monitoring_Admin"}],"teams":["f8c9079d-c7bb-4e58-ac83-359cb217a3b5"],"actions":[],"priority":"P3","oldPriority":"P3","source":"Splunk"},"customerConsolidated":false,"customerId":"3a1f4387-b87b-4a3a-a568-cc372a86d8e4","action":"Close","mappedActionDto":{"mappedAction":"postActionToOEC","extraField":""},"alertId":"913a3db5-7e2a-44f4-a4ff-3002af480c8d-1731522737697","alertAlias":"FL-NS-VPX-INT-1|mobileapist?vnetapis003?8777,","alertDetails":{"Raw":"","Results Link":"https://hostname:8000/app/search/search?q=%7Cloadjob%20scheduler__td26605__search__RMD5e461b39d4ff19795_at_1731522600_38116%20%7C%20head%204%20%7C%20tail%201&earliest=0&latest=now","SuppressClosed":"True","TeamsDescription":"True"},"entity":{"alertId":"913a3db5-7e2a-44f4-a4ff-3002af480c8d-1731522737697","id":"913a3db5-7e2a-44f4-a4ff-3002af480c8d-1731522737697","type":"alert","message":"[Splunk] Load Balancer Member Status","tags":[],"tinyId":"14585","entity":"","alias":"FL-NS-VPX-INT-1|mobileapist?vnetapis003?8777,","createdAt":1731522737697,"updatedAt":1731523038582000000,"username":"System","responders":[{"id":"f8c9079d-c7bb-4e58-ac83-359cb217a3b5","type":"team","name":"Monitoring_Admin"}],"teams":["f8c9079d-c7bb-4e58-ac83-359cb217a3b5"],"actions":[],"priority":"P3","oldPriority":"P3","source":"Splunk"}} [36mmessageId[0m=7546739e-2bab-414d-94b5-b0f205208932

Thank you for all the help on this one,

Thanks,

Tom

Re: Need help with the correct regex

MuS — Wed, 13 Nov 2024 22:08:27 GMT

Hi there,

Can you please post an example _raww event in a code block, thanks

cheers, MuS

Re: Need help with the correct regex

yuanliu — Thu, 14 Nov 2024 04:59:09 GMT

You are looking at the wrong tool in the box. Do not use rex to extract fields from structure data like JSON which your event contains. Instead, extract the JSON object then use tools like spath to extract data fields.

| rex "^[^{]+(?<message_body>.+})" | spath input=message_body | table *.alias *.responders{}.name

Your sample data will give

alert.alias	entity.alias	params.alert.alias	params.entity.alias	alert.responders{}.name	entity.responders{}.name	params.alert.responders{}.name	params.entity.responders{}.name
FL-NS-VPX-INT-1\|mobileapist?vnetapis003?8777,	FL-NS-VPX-INT-1\|mobileapist?vnetapis003?8777,	FL-NS-VPX-INT-1\|mobileapist?vnetapis003?8777,	FL-NS-VPX-INT-1\|mobileapist?vnetapis003?8777,	Monitoring_Admin	Monitoring_Admin	Monitoring_Admin	Monitoring_Admin

Additional pointers:

The sample JSON contains 4 different leaf nodes all named alias. There is no inherent logic to say they are all the same.
The sample JSON contains 4 different arrays that all contain leaf nodes that are all named name. There is no inherent logic to say they are all the same.

What this means is that you need to ask your developer which node you need data from.

Lastly, this JSON has a deep structure. If you are only interested in select few nodes, you can also use a JSON function if your server is 8.2 or later. For example,

| rex "^[^{]+(?<message_body>.+})" | eval alias = json_extract(message_body, "alert.alias"), name = json_extract(message_body, "alert.responders{}.name") | table alias name

The output will be

alias	name
FL-NS-VPX-INT-1\|mobileapist?vnetapis003?8777,	Monitoring_Admin

Here is an emulation of your sample data. Play with it and compare with real data

| makeresults | eval _raw = "[36mINFO[0m[2024-11-13T13:37:23.9114215-05:00] Message body: {\"actionType\":\"custom\",\"customerId\":\"3a1f4387-b87b-4a3a-a568-cc372a86d8e4\",\"ownerDomain\":\"integration\",\"ownerId\":\"8b500163-8476-4b0e-9ef7-2cfdaa272adf\",\"discardScriptResponse\":true,\"sendCallbackToStreamHub\":false,\"requestId\":\"18dcdb1b-14d6-4b10-ad62-3f73acaaef2a\",\"action\":\"Close\",\"productSource\":\"Opsgenie\",\"customerDomain\":\"siteone\",\"integrationName\":\"Opsgenie Edge Connector\",\"integrationId\":\"8b500163-8476-4b0e-9ef7-2cfdaa272adf\",\"customerTransitioningOrConsolidated\":false,\"source\":{\"name\":\"\",\"type\":\"system\"},\"type\":\"oec\",\"receivedAt\":1731523037863,\"ownerId\":\"8b500163-8476-4b0e-9ef7-2cfdaa272adf\",\"params\":{\"type\":\"oec\",\"alertId\":\"913a3db5-7e2a-44f4-a4ff-3002af480c8d-1731522737697\",\"customerId\":\"3a1f4387-b87b-4a3a-a568-cc372a86d8e4\",\"action\":\"Close\",\"integrationId\":\"8b500163-8476-4b0e-9ef7-2cfdaa272adf\",\"integrationName\":\"Opsgenie Edge Connector\",\"integrationType\":\"OEC\",\"customerDomain\":\"siteone\",\"alertDetails\":{\"Raw\":\"\",\"Results Link\":\"https://hostname:8000/app/search/search?q=%7Cloadjob%20scheduler__td26605__search__RMD5e461b39d4ff19795_at_1731522600_38116%20%7C%20head%204%20%7C%20tail%201&earliest=0&latest=now\",\"SuppressClosed\":\"True\",\"TeamsDescription\":\"True\"},\"alertAlias\":\"FL-NS-VPX-INT-1|mobileapist?vnetapis003?8777,\",\"receivedAt\":1731523037863,\"customerConsolidated\":false,\"customerTransitioningOrConsolidated\":false,\"productSource\":\"Opsgenie\",\"source\":{\"name\":\"\",\"type\":\"system\"},\"alert\":{\"alertId\":\"913a3db5-7e2a-44f4-a4ff-3002af480c8d-1731522737697\",\"id\":\"913a3db5-7e2a-44f4-a4ff-3002af480c8d-1731522737697\",\"type\":\"alert\",\"message\":\"[Splunk] Load Balancer Member Status\",\"tags\":[],\"tinyId\":\"14585\",\"entity\":\"\",\"alias\":\"FL-NS-VPX-INT-1|mobileapist?vnetapis003?8777,\",\"createdAt\":1731522737697,\"updatedAt\":1731523038582000000,\"username\":\"System\",\"responders\":[{\"id\":\"f8c9079d-c7bb-4e58-ac83-359cb217a3b5\",\"type\":\"team\",\"name\":\"Monitoring_Admin\"}],\"teams\":[\"f8c9079d-c7bb-4e58-ac83-359cb217a3b5\"],\"actions\":[],\"priority\":\"P3\",\"oldPriority\":\"P3\",\"source\":\"Splunk\"},\"entity\":{\"alertId\":\"913a3db5-7e2a-44f4-a4ff-3002af480c8d-1731522737697\",\"id\":\"913a3db5-7e2a-44f4-a4ff-3002af480c8d-1731522737697\",\"type\":\"alert\",\"message\":\"[Splunk] Load Balancer Member Status\",\"tags\":[],\"tinyId\":\"14585\",\"entity\":\"\",\"alias\":\"FL-NS-VPX-INT-1|mobileapist?vnetapis003?8777,\",\"createdAt\":1731522737697,\"updatedAt\":1731523038582000000,\"username\":\"System\",\"responders\":[{\"id\":\"f8c9079d-c7bb-4e58-ac83-359cb217a3b5\",\"type\":\"team\",\"name\":\"Monitoring_Admin\"}],\"teams\":[\"f8c9079d-c7bb-4e58-ac83-359cb217a3b5\"],\"actions\":[],\"priority\":\"P3\",\"oldPriority\":\"P3\",\"source\":\"Splunk\"},\"mappedActionDto\":{\"mappedAction\":\"postActionToOEC\",\"extraField\":\"\"},\"ownerId\":\"8b500163-8476-4b0e-9ef7-2cfdaa272adf\"},\"integrationType\":\"OEC\",\"alert\":{\"alertId\":\"913a3db5-7e2a-44f4-a4ff-3002af480c8d-1731522737697\",\"id\":\"913a3db5-7e2a-44f4-a4ff-3002af480c8d-1731522737697\",\"type\":\"alert\",\"message\":\"[Splunk] Load Balancer Member Status\",\"tags\":[],\"tinyId\":\"14585\",\"entity\":\"\",\"alias\":\"FL-NS-VPX-INT-1|mobileapist?vnetapis003?8777,\",\"createdAt\":1731522737697,\"updatedAt\":1731523038582000000,\"username\":\"System\",\"responders\":[{\"id\":\"f8c9079d-c7bb-4e58-ac83-359cb217a3b5\",\"type\":\"team\",\"name\":\"Monitoring_Admin\"}],\"teams\":[\"f8c9079d-c7bb-4e58-ac83-359cb217a3b5\"],\"actions\":[],\"priority\":\"P3\",\"oldPriority\":\"P3\",\"source\":\"Splunk\"},\"customerConsolidated\":false,\"customerId\":\"3a1f4387-b87b-4a3a-a568-cc372a86d8e4\",\"action\":\"Close\",\"mappedActionDto\":{\"mappedAction\":\"postActionToOEC\",\"extraField\":\"\"},\"alertId\":\"913a3db5-7e2a-44f4-a4ff-3002af480c8d-1731522737697\",\"alertAlias\":\"FL-NS-VPX-INT-1|mobileapist?vnetapis003?8777,\",\"alertDetails\":{\"Raw\":\"\",\"Results Link\":\"https://hostname:8000/app/search/search?q=%7Cloadjob%20scheduler__td26605__search__RMD5e461b39d4ff19795_at_1731522600_38116%20%7C%20head%204%20%7C%20tail%201&earliest=0&latest=now\",\"SuppressClosed\":\"True\",\"TeamsDescription\":\"True\"},\"entity\":{\"alertId\":\"913a3db5-7e2a-44f4-a4ff-3002af480c8d-1731522737697\",\"id\":\"913a3db5-7e2a-44f4-a4ff-3002af480c8d-1731522737697\",\"type\":\"alert\",\"message\":\"[Splunk] Load Balancer Member Status\",\"tags\":[],\"tinyId\":\"14585\",\"entity\":\"\",\"alias\":\"FL-NS-VPX-INT-1|mobileapist?vnetapis003?8777,\",\"createdAt\":1731522737697,\"updatedAt\":1731523038582000000,\"username\":\"System\",\"responders\":[{\"id\":\"f8c9079d-c7bb-4e58-ac83-359cb217a3b5\",\"type\":\"team\",\"name\":\"Monitoring_Admin\"}],\"teams\":[\"f8c9079d-c7bb-4e58-ac83-359cb217a3b5\"],\"actions\":[],\"priority\":\"P3\",\"oldPriority\":\"P3\",\"source\":\"Splunk\"}} [36mmessageId[0m=7546739e-2bab-414d-94b5-b0f205208932" ``` data emulation above ```