I am trying to extract 2 fields from my logs.
Logs:
10.218.136.20 - - [30/Jun/2023:02:36:32 +0000] "GET /api/v2/runs/run-g1mhsXooK6aKV9bS?include=plan%2Ccost_estimate%2Capply%2Ccreated_by HTTP/1.1" 200 5460 "https://terraform.srv.companyname.com.au/app/customer/workspaces/a00ccc-tfe-test02-customer_infra_ping/runs/run-g1mhsXooK6aKV9bS" "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/114.0.0.0 Safari/537.36"
Here i want to extract 2 new fields
1. workspace_name="a00ccc-tfe-test02-customer_infra_ping"
2. workspace_id="g1mhsXooK6aKV9bS"
Please help me with regex & thanks in advance!
This rex shouldn't match this first occurrence of runs/run-xxx as it expecting that there must be workspaces/xxxx before it. So it is matching the second
runs/run-Y63d5qeBk3pDHpJZ"
which didn't contains / character. BTW there was mistake \/ instead of \\ on rex. I have fixed it already, so you should also fix it in your extractions.
Can you test this without extractions just with SPL?
Something like
index=test sourcetype="testdata"
| rex "workspaces\/(?<workspace_name>[^\/]+)\/runs\/run-(?<workspace_id>[^\"\\]+)"
| table _time workspace_name workspace_id
just check that this works also on your real data. If need you could change the workspace_id to AAAAAA on rex to be sure that it's not defined somewhere else.
Are you sure that you haven't earlier definition for workspace_id somewhere? If you could login to command line you could try
splunk btool props list --debug testdata
This should show all props definition to that sourcetype and where those are defined.
Hi
you could try
rex "workspaces\/(?<workspace_name>[^\/]+)\/runs\/run-(?<workspace_id>[^\"]+)"
see https://regex101.com/r/aywynb/1
r. Ismo
@isoutamo Thanks for your reply.
Results for Workspace_name is fine. But workspace_id ends with a backslash. Can i get the result without \
Your examples didn' t contains / on workspace_id! You could try this
rex "workspaces\/(?<workspace_name>[^\/]+)\/runs\/run-(?<workspace_id>[^\"\\]+)"
@isoutamo Still i am getting "\" at the end of the workspace_id results.
Then your logs are not as your examples was on this thread. Can you give some more events as an example? Especially those with has the \ at the end on w_id!
Other logs
10.218.136.20 - - [30/Jun/2023:02:36:37 +0000] "GET /api/v2/workspaces/ws-ukz9TnHNE9kN4eCa?include=agent_pool%2Ccurrent_configuration_version%2Ccurrent_run%2Ccurrent_state_version%2Clocked_by%2Creadme%2Coutputs HTTP/1.1" 304 0 "https://terraform.srv.companyname.com.au/app/customer/workspaces/a00ccc-tfe-dev02-customer_infra/runs/run-ACevPzmMTYE6UP5e" "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/114.0.0.0 Safari/537.36"
10.218.136.20 - - [30/Jun/2023:02:36:40 +0000] "GET /api/v2/runs/run-Y63d5qeBk3pDHpJZ/run-events?include=comment%2Cactor HTTP/1.1" 304 0 "https://terraform.srv.companyname.com.au/app/customer/workspaces/a00964-tfe-test-customer_infra_main/runs/run-Y63d5qeBk3pDHpJZ" "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/114.0.0.0 Safari/537.36"
Very interesting. As you can see here https://regex101.com/r/0gc9lk/1 those are extracted correctly. Are you sure that you don't add that \ to this string? On your examples there are no \ character at the end of n_id!
Can you show your SPL?
First line of the below log Y63......ends with /. Whereas third line y63....ends usually. Does this make change?
10.218.136.20 - - [30/Jun/2023:02:36:40 +0000] "GET /api/v2/runs/run-Y63d5qeBk3pDHpJZ/run-events?include=comment%2Cactor HTTP/1.1" 304 0 "https://terraform.srv.companyname.com.au/app/customer/workspaces/a00964-tfe-test-customer_infra_main/runs/run-Y63d5qeBk3pDHpJZ" "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/114.0.0.0 Safari/537.36"
This rex shouldn't match this first occurrence of runs/run-xxx as it expecting that there must be workspaces/xxxx before it. So it is matching the second
runs/run-Y63d5qeBk3pDHpJZ"
which didn't contains / character. BTW there was mistake \/ instead of \\ on rex. I have fixed it already, so you should also fix it in your extractions.
Can you test this without extractions just with SPL?
Something like
index=test sourcetype="testdata"
| rex "workspaces\/(?<workspace_name>[^\/]+)\/runs\/run-(?<workspace_id>[^\"\\]+)"
| table _time workspace_name workspace_id
just check that this works also on your real data. If need you could change the workspace_id to AAAAAA on rex to be sure that it's not defined somewhere else.
Are you sure that you haven't earlier definition for workspace_id somewhere? If you could login to command line you could try
splunk btool props list --debug testdata
This should show all props definition to that sourcetype and where those are defined.