Knowledge Management

how do I remove the single quotes from my key value pairs,how to remove single quotes from a key value pair

New Member

I have a logs like below and this is not a JSON logs, indexing through HEC.

Key1='value1' Key2='value'

how do I remove this single quotes from value?

Regards,
Thippesh

0 Karma
1 Solution

Super Champion

you can do in few ways
1. At indextime, completely remove this before indexing. May not be good if you want to keep "pure" source data
2. replace all single quotes in _raw message using rex | rex mode=sed "y/\=\'/\=/"| rex mode=sed "y/\'\s/\s/"
3. Do individual replacement of key-value | eval key1=replace(key1,"\'"."")

View solution in original post

0 Karma

Esteemed Legend

Assuming that search-time is OK, like this:

|makeresults | eval "'foo'" = "'bar'"

| rename COMMENT AS "Everything above generates sample event data; everything below is your solution"

| foreach "'*'" [rex mode=sed field="<<FIELD>>" "s/^'// s/'$//"]
| rename "'*'" AS "*"
0 Karma

New Member

Hi woodcock,

Thanks for the answer, would like to know how I can fix this in index time so that I no need to do this for each search.

0 Karma

Esteemed Legend

In order to do that, you MUST show us a sample event. Why do OPs so rarely share sample events?

0 Karma

Super Champion

you can do in few ways
1. At indextime, completely remove this before indexing. May not be good if you want to keep "pure" source data
2. replace all single quotes in _raw message using rex | rex mode=sed "y/\=\'/\=/"| rex mode=sed "y/\'\s/\s/"
3. Do individual replacement of key-value | eval key1=replace(key1,"\'"."")

View solution in original post

0 Karma

New Member

Hi koshyk,

Thanks for the answer, would like to know how I can fix this in index time so that I no need to do this for each search.

0 Karma

Super Champion

If you want in indextime, you can do in two ways

  1. Just using props.conf

    [myQuoteSourcetype]
    SEDCMD-singleQuotedData = s/(\w+)=\'([^\']+)/\1=\2/g

Please try and test your regex properly

  1. Using props and Transforms (In case if you want to re-use in multiple sourcetypes)

In props.conf

[myQuoteSourcetype]
TRANSFORMS-removeSingleQuotedKeys = removeSingleQuotedKeys

In transforms.conf

[removeSingleQuotedKeys]
REGEX = (\w+)=(?:\'?)([^\']+)(?:\'?)\s
FORMAT = $1:$2\s
DEST_KEY = _raw

I've tested sample data Regex I've put into below link:
https://regex101.com/r/4caTBy/1

But of course, you need to fine tune your regex

(Please upvote/accept, if it worked. Cheers)

0 Karma