topic Re: How to break an incoming event into searchable fields? in Splunk Search

How to break an incoming event into searchable fields?

lennys26 — Mon, 08 Sep 2014 14:40:19 GMT

I am struggling to figure out how to break an incoming event into [searchable] fields and am hoping someone could point me in the right direction.

See my data below which is received as you see it and without an index line, per se. I have been reading the online docs, answers, investigating props.conf and playing with regex to pull out the data fields, but am not getting anywhere. Can someone assist?

The fields that I want to utilze are from "INCATOT" through "MIDFAIL" and the corresponding values are "31614" through "0", however initially I could work with a smaller set of fields. I presume that I could use the "INCATOT" -> "MIDFAIL" text as the log index, but I really could use some assistance here.

Thanks in advance.

AMSTNLA201A CM           OMPR201 SEP08 15:16:04 3202 INFO OM REPORT 
    CLASS:   LCR_OM
    START:2014/09/08 15:00:00 MON; STOP: 2014/09/08 15:15:00 MON;
    SLOWSAMPLES:         9 ; FASTSAMPLES:         90 ;

    TRK
          KEY (COMMON_LANGUAGE_NAME)
          INFO (OM2TRKINFO)
             INCATOT   PRERTEAB     INFAIL    NATTMPT   NOVFLATB      GLARE    OUTFAIL    DEFLDCA       DREU       PREU
                 TRU        SBU        MBU   OUTMTCHF    CONNECT     TANDEM        AOF        ANF       TOTU     ANSWER
             ACCCONG   NOANSWER   INANSWER    OUTANSU     INANSU    MIDFAIL

         --------------------------------------------------------------------------------------------------------------
               31614         31         84      35822       1351          4         16          0          0          0
              116631          0        397          0      34402      34255          0          0     117028      26421
                   0          0          0          0          0          0

Re: How to break an incoming event into searchable fields?

tom_frotscher — Mon, 08 Sep 2014 14:52:35 GMT

Is the sequence of your field names (INCATOT to MIDFAIL) fixed? Or can it change from event to event?

Re: How to break an incoming event into searchable fields?

kml_uvce — Mon, 08 Sep 2014 15:00:06 GMT

you can do like this.
1)if your events are having fixed INCATOT to MIDFAIL then break your event starting from INCATOT
2)if your events are not having then break events starting from digit newline then word

Then you can extract fields by using regex.

Re: How to break an incoming event into searchable fields?

lennys26 — Mon, 08 Sep 2014 15:11:33 GMT

The field names (INCATOT to MIDFAIL) are fixed.

Re: How to break an incoming event into searchable fields?

HansK — Fri, 19 Sep 2014 12:15:00 GMT

This will do the trick:

 [\n\r].*--\s*(?<INCATOT>[0-9]*)\s*(?<PRERTEAB>[0-9]*)\s*(?<INFAIL>[0-9]*)\s*(?<NATTMPT>[0-9]*)\s*(?<NOVFLATB>[0-9]*)\s*([0-9]*)\s*([0-9]*)\s*([0-9]*)\s*([0-9]*)\s*([0-9]*)\s*([0-9]*)\s*([0-9]*)\s*([0-9]*)\s*([0-9]*)\s*([0-9]*)\s*([0-9]*)\s*([0-9]*)\s*([0-9]*)\s*([0-9]*)\s*([0-9]*)\s*([0-9]*)

Re: How to break an incoming event into searchable fields?

lennys26 — Thu, 30 Oct 2014 20:54:34 GMT

I just wanted to post what I have ended up using as the solution.

| rex field=_raw "-+[^\\d]+(?<INCATOT>[^ ]+)\\s+(?<PRERTEAB>[^ ]+)\\s+(?<INFAIL>[^ ]+)\\s+(?<NATTMPT>[^ ]+)\\s+(?<NOVFLATB>[^ ]+)\\s+(?<GLARE>[^ ]+)\\s+(?<OUTFAIL>[^ ]+)\\s+(?<DEFLDCA>[^ ]+)\\s+(?<DREU>[^ ]+)\\s+(?<PREU>[^ ]+)\\s+(?<TRU>[^ ]+)\\s+(?<SBU>[^ ]+)\\s+(?<MBU>[^ ]+)\\s+(?<OUTMTCHF>[^ ]+)\\s+(?<CONNECT>[^ ]+)\\s+(?<TANDEM>[^ ]+)\\s+(?<AOF>[^ ]+)\\s+(?<ANF>[^ ]+)\\s+(?<TOTU>[^ ]+)\\s+(?<ANSWER>[^ ]+)\\s+(?<ACCCONG>[^ ]+)\\s+(?<NOANSWER>[^ ]+)\\s+(?<INANSWER>[^ ]+)\\s+(?<OUTANSU>[^ ]+)\\s+(?<INANSU>[^ ]+)\\s+(?<MIDFAIL>[^ ]+)"