Can splunk do this? Extract a portion of an event. Then, index just the value and not the key=value pairs.

[lpolo]

I have the following log event :

2013-03-12 10:37:10,205 { "start" : 1, "returned" : 1, "count" : 1, "entities" : [ { "houses" : { "callers" : "IM", "placeid" : 5041447014850446107, "number" : 14, "sourceid" : 5625 }, "entitytype" : "house/street", "title" : [ { "default" : "No Place" } ] } ] }

With the following query:

| rex field=_raw mode=sed "s/[rn]//g" 
| rex "<tvsquery id="(?&lt;id">[^>]+)>(?<response_temp>.+?)</tvsquery>"
| eval result="{\""."TimeStamp"."\"".":"."\""._time."\"".","."\""."id"."\"".":"."\"".id."\"".","."\""."response"."\"".":".response_temp."}" |table result

I am able to get the following result set:

result
{"TimeStamp":"1364556697.631","id":"58b6bf4d-948b-416b-8d17-cedcbc1059ec","response":{ "start" : 1, "returned" : 1, "count" : 1, "entities" : [ { "houses" : { "callers" : "IM", "placeid" : 5041447014850446107, "number" : 14, "sourceid" : 5625 }, "entitytype" : "house/street", "title" : [ { "default" : "No Place" } ] } ] }}

Problem:
How can index the value of result in a new summary index?
For this particular example I just need this:

{"TimeStamp":"1364556697.631","id":"58b6bf4d-948b-416b-8d17-cedcbc1059ec","response":{ "start" : 1, "returned" : 1, "count" : 1, "entities" : [ { "houses" : { "callers" : "IM", "placeid" : 5041447014850446107, "number" : 14, "sourceid" : 5625 }, "entitytype" : "house/street", "title" : [ { "default" : "No Place" } ] } ] }}

I do not want this in the index:

result = "{"TimeStamp":"1364556697.631","id":"58b6bf4d-948b-416b-8d17-cedcbc1059ec","response":{ "start" : 1, "returned" : 1, "count" : 1, "entities" : [ { "houses" : { "callers" : "IM", "placeid" : 5041447014850446107, "number" : 14, "sourceid" : 5625 }, "entitytype" : "house/street", "title" : [ { "default" : "No Place" } ] } ] }}"

Thanks,
Lp

[yannK]

to replace the _raw event, try

| eval _raw="< my long eval statement >"

to summarize the result, look at the |collect function.

[lpolo]

It worked. I forgot about _raw.

Thanks.

[lpolo]

I am not sure If I could use transforms.conf to transform an event that is already indexed. I have never done it before.

I have used transform.conf to transform an event that is going to be indexed for the first time.

Could you provide an example based on above question?

Thanks,
Lp

[krugger]

Don't quite understand the problem. Splunk will generate a table and each column has a name.

So you getting:

result

{"TimeStamp":"1364556697.631","id":"58b6bf4d-948b-416b-8d17-cedcbc1059ec","response":{ "start" : 1, "returned" : 1, "count" : 1, "entities" : [ { "houses" : { "callers" : "IM", "placeid" : 5041447014850446107, "number" : 14, "sourceid" : 5625 }, "entitytype" : "house/street", "title" : [ { "default" : "No Place" } ] } ] }}

You seem to be getting exactly what you are looking for.

If you want to index it, you should consider using a transform to extract the fields.