Splunk Search

Unable to extract two different pattern strings by rex command

bhaskar3107
New Member

{"timestamp":"2021-02-24T00:00:46.533+00:00","message":"Snapshot event published: SnapshotEvent(status=CREATED, version=SnapshotVersion(sourceSystem=zvkk, source=sdp/deposits/zvkk/2021-02-24/NextBusinessDays/Snapshot1, entityType=NEXT_BUSINESS_DAYS, date=2021-02-24, version=1, snapshotSize=5, uuid=8683aa33-3a6c-4087-9cdd-3084d8e70147, holiday=false))","component":"com.db.sdda.dc.kafka.service.SnapshotEventNotifyService","thread":"scheduling-1","level":"INFO"}

 

{"timestamp":"2021-02-23T20:56:37.797+00:00","message":"Snapshot event published: SnapshotEvent(status=CREATED, version=SnapshotVersion(sourceSystem=IDMS-0781, source=sdp/deposits/IDMS-0781/2021-02-23/FacilityLimit/Snapshot1, entityType=FACILITY, date=2021-02-23, version=1, snapshotSize=15168, uuid=016cc1ad-8c27-4144-a9d2-c0233cc1e450, holiday=false))","component":"com.db.sdda.dc.kafka.service.SnapshotEventNotifyService","thread":"scheduling-1","level":"INFO"}

 

Used command below 
|rex field=_raw "sourceSystem=(?<So1>\w+[-]\w+)"    --> Able to get IDMS-0781 as a output but unable to get single word branch like zvkk 
|rex field=_raw "sourceSystem=(?<So2>\w+)"

Problem Statement
1.I would like to extract sourcesystem in way everything before comma (SourceSystem=IDMS-0781,)
2.Or both with hyphen and without hyphen should pick by rex command 

 

Labels (1)
0 Karma

jmartens
Path Finder

Assuming the field is always starting with 'SourceSystem=' and ending with  a comma (,) you could do this:

 

| rex field=_raw "sourceSystem=(?<field_name>[^,]+)"

 

But as additional advice it seems your data is JSON you might be better of using spath to extract your field(s) or even better configure your sourcetype correctly using props.conf and

INDEXED_EXTRACTIONS=json

or

KV_MODE=json

For details see the Getting data in primer especially Extract fields from files with structured data.

As an example of what you could do here is a run very-where example:

 

| makeresults count=2 
| streamstats count 
| eval _raw=case(count=1, "{\"timestamp\":\"2021-02-24T00:00:46.533+00:00\",\"message\":\"Snapshot event published: SnapshotEvent(status=CREATED, version=SnapshotVersion(sourceSystem=zvkk, source=sdp\/deposits\/zvkk\/2021-02-24\/NextBusinessDays\/Snapshot1, entityType=NEXT_BUSINESS_DAYS, date=2021-02-24, version=1, snapshotSize=5, uuid=8683aa33-3a6c-4087-9cdd-3084d8e70147, holiday=false))\",\"component\":\"com.db.sdda.dc.kafka.service.SnapshotEventNotifyService\",\"thread\":\"scheduling-1\",\"level\":\"INFO\"}", count=2, "{\"timestamp\":\"2021-02-23T20:56:37.797+00:00\",\"message\":\"Snapshot event published: SnapshotEvent(status=CREATED, version=SnapshotVersion(sourceSystem=IDMS-0781, source=sdp\/deposits\/IDMS-0781\/2021-02-23\/FacilityLimit\/Snapshot1, entityType=FACILITY, date=2021-02-23, version=1, snapshotSize=15168, uuid=016cc1ad-8c27-4144-a9d2-c0233cc1e450, holiday=false))\",\"component\":\"com.db.sdda.dc.kafka.service.SnapshotEventNotifyService\",\"thread\":\"scheduling-1\",\"level\":\"INFO\"}") 
| spath path=message
| eval backup=_raw
| eval _raw=message
| extract
| eval _raw=backup
| rename backup as _raw

 

The first lines are to mimic your example, then there is a spath extraction (since I did not bother creating a proper sourcetype configuration) to extract the message part using:

 

| spath path=message

 

 Then we need to do some backing up of the raw event as extract only operates on the raw event so we copy _raw to backup, assign message to _raw:

 

| rename _raw as backup message as _raw

 

Now we can run an extract command as you data is already in the default key = value pair format:

 

| extract

 

And then revert our rename command to have our original values again:

 

| rename _raw as message backup as _raw

 

0 Karma
Get Updates on the Splunk Community!

Introducing the Splunk Community Dashboard Challenge!

Welcome to Splunk Community Dashboard Challenge! This is your chance to showcase your skills in creating ...

Get the T-shirt to Prove You Survived Splunk University Bootcamp

As if Splunk University, in Las Vegas, in-person, with three days of bootcamps and labs weren’t enough, now ...

Wondering How to Build Resiliency in the Cloud?

IT leaders are choosing Splunk Cloud as an ideal cloud transformation platform to drive business resilience,  ...