Splunk Search

Limit in manual editing regex

javo
Explorer

Hello there

What I'm doing is extracting fields from my log file and every entry has about 20 fields separated by commas, some of them empty. Something like this:

FULL,34,32136783,2012-03-07 14:23:43,120,,0,4|0|65|1|02,no_failure,,,,,,,no_text,10%,145432236,15-47658995,-1

(Please note the repeated commas twice, that's not a typing mistake. Those are empty fields)

It happens that I need to edit manualy the regex because the automatic recognition doesn't work well, but the windows dialog that opens by clicking 'Edit' doesn't let me write the very long regular expression I need. I can write exactly 200 characters and my regex is at least two times longer. And there you go:

^(?P<FIELDNAME1>[^,]+),(?P<FIELDNAME2>[^,]+),(?P<FIELDNAME3>[^,]+),... and so on 20 times

So, does anyone know how to solve this inconvenient? Another way to tell Splunk to extract those 20 fields? Or perhaps an equivalent, shorter regex?

Thank you whoever helps!

Tags (3)
0 Karma
1 Solution

sbrant_splunk
Splunk Employee
Splunk Employee

Using props and transforms in this case is the right thing to do, if you can. An alternative, if you were to not have access, may be to use multiple extractions to get all of the fields, using a regex such as this:

^(?:(?<field1>[^,]*),){1}             # catches the first field
^(?:(?<field2>[^,]*),){2}             # catches the second field
.
.
.
^(?:(?<field20>[^,]*),){20}            # 20

View solution in original post

0 Karma

sbrant_splunk
Splunk Employee
Splunk Employee

Using props and transforms in this case is the right thing to do, if you can. An alternative, if you were to not have access, may be to use multiple extractions to get all of the fields, using a regex such as this:

^(?:(?<field1>[^,]*),){1}             # catches the first field
^(?:(?<field2>[^,]*),){2}             # catches the second field
.
.
.
^(?:(?<field20>[^,]*),){20}            # 20
0 Karma

Drainy
Champion

Props.conf
[host::foo]
TRANSFORMS-foo=netscreen-error-field

Transforms.conf
[netscreen-error-field]
REGEX = device_id=[w+](?[^:]+)
FORMAT = err_code::$1

I've just edited one part from the docs. They are written by users so they are what a regular user would write.

0 Karma

javo
Explorer

I don't know, I just want to see something.

0 Karma

Ayn
Legend

What are you missing from the examples? They look very much like what I would write something as a 'regular user'.

javo
Explorer

Could anyone show me a user-made props.conf, or at least a piece of it?
I know there is an example in Splunk's directory but it would be much better for me to see something made by a user.

0 Karma

Ayn
Legend

Do you have shell access on the Splunk server? Because what you do in the field extractor in the web interface ultimately ends up in a props.conf / transforms.conf file. By editing that file directly you won't have any problems with limits like this.

javo
Explorer

Yes, I do have access. Should I edit both files?

0 Karma
Get Updates on the Splunk Community!

Index This | I am a number, but when you add ‘G’ to me, I go away. What number am I?

March 2024 Edition Hayyy Splunk Education Enthusiasts and the Eternally Curious!  We’re back with another ...

What’s New in Splunk App for PCI Compliance 5.3.1?

The Splunk App for PCI Compliance allows customers to extend the power of their existing Splunk solution with ...

Extending Observability Content to Splunk Cloud

Register to join us !   In this Extending Observability Content to Splunk Cloud Tech Talk, you'll see how to ...