Splunk Search

Unable to extract two different pattern strings by rex command

bhaskar3107
New Member

{"timestamp":"2021-02-24T00:00:46.533+00:00","message":"Snapshot event published: SnapshotEvent(status=CREATED, version=SnapshotVersion(sourceSystem=zvkk, source=sdp/deposits/zvkk/2021-02-24/NextBusinessDays/Snapshot1, entityType=NEXT_BUSINESS_DAYS, date=2021-02-24, version=1, snapshotSize=5, uuid=8683aa33-3a6c-4087-9cdd-3084d8e70147, holiday=false))","component":"com.db.sdda.dc.kafka.service.SnapshotEventNotifyService","thread":"scheduling-1","level":"INFO"}

 

{"timestamp":"2021-02-23T20:56:37.797+00:00","message":"Snapshot event published: SnapshotEvent(status=CREATED, version=SnapshotVersion(sourceSystem=IDMS-0781, source=sdp/deposits/IDMS-0781/2021-02-23/FacilityLimit/Snapshot1, entityType=FACILITY, date=2021-02-23, version=1, snapshotSize=15168, uuid=016cc1ad-8c27-4144-a9d2-c0233cc1e450, holiday=false))","component":"com.db.sdda.dc.kafka.service.SnapshotEventNotifyService","thread":"scheduling-1","level":"INFO"}

 

Used command below 
|rex field=_raw "sourceSystem=(?<So1>\w+[-]\w+)"    --> Able to get IDMS-0781 as a output but unable to get single word branch like zvkk 
|rex field=_raw "sourceSystem=(?<So2>\w+)"

Problem Statement
1.I would like to extract sourcesystem in way everything before comma (SourceSystem=IDMS-0781,)
2.Or both with hyphen and without hyphen should pick by rex command 

 

Labels (1)
0 Karma

jmartens
Path Finder

Assuming the field is always starting with 'SourceSystem=' and ending with  a comma (,) you could do this:

 

| rex field=_raw "sourceSystem=(?<field_name>[^,]+)"

 

But as additional advice it seems your data is JSON you might be better of using spath to extract your field(s) or even better configure your sourcetype correctly using props.conf and

INDEXED_EXTRACTIONS=json

or

KV_MODE=json

For details see the Getting data in primer especially Extract fields from files with structured data.

As an example of what you could do here is a run very-where example:

 

| makeresults count=2 
| streamstats count 
| eval _raw=case(count=1, "{\"timestamp\":\"2021-02-24T00:00:46.533+00:00\",\"message\":\"Snapshot event published: SnapshotEvent(status=CREATED, version=SnapshotVersion(sourceSystem=zvkk, source=sdp\/deposits\/zvkk\/2021-02-24\/NextBusinessDays\/Snapshot1, entityType=NEXT_BUSINESS_DAYS, date=2021-02-24, version=1, snapshotSize=5, uuid=8683aa33-3a6c-4087-9cdd-3084d8e70147, holiday=false))\",\"component\":\"com.db.sdda.dc.kafka.service.SnapshotEventNotifyService\",\"thread\":\"scheduling-1\",\"level\":\"INFO\"}", count=2, "{\"timestamp\":\"2021-02-23T20:56:37.797+00:00\",\"message\":\"Snapshot event published: SnapshotEvent(status=CREATED, version=SnapshotVersion(sourceSystem=IDMS-0781, source=sdp\/deposits\/IDMS-0781\/2021-02-23\/FacilityLimit\/Snapshot1, entityType=FACILITY, date=2021-02-23, version=1, snapshotSize=15168, uuid=016cc1ad-8c27-4144-a9d2-c0233cc1e450, holiday=false))\",\"component\":\"com.db.sdda.dc.kafka.service.SnapshotEventNotifyService\",\"thread\":\"scheduling-1\",\"level\":\"INFO\"}") 
| spath path=message
| eval backup=_raw
| eval _raw=message
| extract
| eval _raw=backup
| rename backup as _raw

 

The first lines are to mimic your example, then there is a spath extraction (since I did not bother creating a proper sourcetype configuration) to extract the message part using:

 

| spath path=message

 

 Then we need to do some backing up of the raw event as extract only operates on the raw event so we copy _raw to backup, assign message to _raw:

 

| rename _raw as backup message as _raw

 

Now we can run an extract command as you data is already in the default key = value pair format:

 

| extract

 

And then revert our rename command to have our original values again:

 

| rename _raw as message backup as _raw

 

0 Karma
Get Updates on the Splunk Community!

Enterprise Security Content Update (ESCU) | New Releases

In December, the Splunk Threat Research Team had 1 release of new security content via the Enterprise Security ...

Why am I not seeing the finding in Splunk Enterprise Security Analyst Queue?

(This is the first of a series of 2 blogs). Splunk Enterprise Security is a fantastic tool that offers robust ...

Index This | What are the 12 Days of Splunk-mas?

December 2024 Edition Hayyy Splunk Education Enthusiasts and the Eternally Curious!  We’re back with another ...