Splunk Search

Regex with multiple fields duplicated. Yet data can be different?

Communicator

I have an issue with data titles that would appear to be repeated, yet in the case below, The passwordexpiry_date: field can be repeated with different values.

See below account fields are replicated and I am trying to determine if a REGEX can even be written on this. Especially when field names are duplicated.

sequencenumber=18112355,
remote
client=Eserver,
2014-05-08 18:02:47:84 GMT,
conn=72276,
op=239,
eventID=0a46b98715e113924708169294656,
messageID=701,
ip=127.0.0.1,
uname=gaspm05,
urole=Bobs And Admin,
msg=Modify user,
msgtype=MODIFY,
result=0,
etime=34ms,
upduserowner=Person User Admin Group,
associatedparentmembergroups=[ ],
upd
user=DATABOBTruck3bobsurunc,
newuser=[userid:
[DATA
BOBTruck3bobsurunc]
firstname:[Truck3bobsurunc]
lastname:[LN]
email:[]
certdn:[]
properties:[[name=ContactID isset=false], [name=ctscPasswordResetAttempts isset=false], [name=ctscSecretQuestionAnswer isset=false]]
account
startdate:[Mar 12, 2014 6:08:47 PM]
account
expirydate:[Jan 1, 4712 6:08:47 PM]
islockedout:[false]
isactive:[true]
passwordexpiry
date:[Aug 6, 2014 6:02:47 PM]
admingroup:[Person User Admin Group]
ispublic:[true]],
original
user=[userid:[DATABOBTruck3bobsurunc]
firstname:[Truck3bobsurunc]
lastname:[LN]
email:[]
certdn:[]
properties:[[name=ContactID isset=false], [name=ctscPasswordResetAttempts isset=false], [name=ctscSecretQuestionAnswer isset=false]]
accountstartdate:[Mar 12, 2014 6:08:47 PM]
accountexpirydate:[Jan 1, 4712 6:08:47 PM]
islockedout:[false]
isactive:[true]
passwordexpirydate:[Jul 30, 2014 6:26:37 PM]
admin
group:[Person User Admin Group]
ispublic:[true]],
userowner=Person User Admin Group,
user=DATA
BOB_Truck3bobsurunc,
userUID=CCCCCQEBBBBDU1FMJJJJBlRlc3R2MgAAAAEQDDDDDDDkjBE=

Tags (3)
0 Karma
1 Solution

Splunk Employee
Splunk Employee
* | rex  max_match=0 "passwordexpiry_date\:\[(?<Password_Expiry_Date>.+?)\]"

View solution in original post

Communicator

Thanks, this is a big help. I will work this into my query. What has been happening is fields that are duplicated just come into the _raw data flow in Splunk. So many good answers. I appreciate every one.

0 Karma

SplunkTrust
SplunkTrust

MV_ADD = true will create a multivalue field rather than ignore duplicate field names.

---
If this reply helps you, an upvote would be appreciated.
0 Karma

Splunk Employee
Splunk Employee
  • | rex maxmatch=0 "passwordexpirydate:[(?.+?)]"

That will give you a field called PasswordExpiryDate with the multiple values per event like you needed. http://docs.splunk.com/Documentation/Splunk/6.0.3/SearchReference/Rex

0 Karma

Communicator

Hi Rich,
Thanks for that information MV_ADD=True so would that allow values to pass through differently?
Unfortunately what I have been given for data has been wrapped by another program. My options are limited. Outside of asking these guys to rewrite their output.

0 Karma

Communicator

Thanks dmaislin this is very helpfull. I will verify it against current output. Looks very promising.

0 Karma

Communicator

Hi Rich,
Thanks for that information MV_ADD=True so would that allow values to pass through differently?
Unfortunately what I have been given for data has been wrapped by another program. My options are limited. Outside of asking these guys to rewrite their output.

0 Karma

Splunk Employee
Splunk Employee
* | rex  max_match=0 "passwordexpiry_date\:\[(?<Password_Expiry_Date>.+?)\]"

View solution in original post

Splunk Employee
Splunk Employee

That will give you a field called PasswordExpiryDate with the multiple values per event like you needed. http://docs.splunk.com/Documentation/Splunk/6.0.3/SearchReference/Rex

0 Karma

SplunkTrust
SplunkTrust

What fields do you want to extract? What should happen with duplicated fields, extract first, extract last, or extract all? Are you sure you need a regex at all? Perhaps adding MV_ADD=true to your props.conf is enough.

---
If this reply helps you, an upvote would be appreciated.
0 Karma

Communicator

To be more clear, I just need to write a Regex on this entre output but I cannot get past the duplicated fields as I am just trying to extract this data with some success. All the REGEX training on the planet may not be enough.

This Data output is ClearTrust. The characters have been changed to conceal identities. The length of data has remained the same however.

0 Karma

SplunkTrust
SplunkTrust

What is it you want regex to do?

---
If this reply helps you, an upvote would be appreciated.
0 Karma