topic Re: How to prevent field names with periods (.) OR hyphens (-) from getting replaced with underscores (_)? in Splunk Search

How to prevent field names with periods (.) OR hyphens (-) from getting replaced with underscores (_)?

jrodriguezap — Sun, 16 Nov 2014 19:11:46 GMT

Hi
I have the following logs:

10/01/2014 00:00:00 -0500, client_host="172.24.1.41", client_id=db01, report_id=RAS04, igoogleinicio.com=3491,  webair.com=13148
10/01/2014 00:00:00 -0500, client_host="172.24.1.41", client_id=db01, report_id=RAS04, smtp.ec.pe=1313,  your-server.de=13148

These data need to view them as a table: ... | table *
And show me the header like this:

client_host  |  client_id  |  report_id  |  igoogleinicio_com  |  webair_com  |  smtp_ec_br  |  your_server_de

As you can see, the fields that have: _ and . are replaced by _ (underline).
There will be no way around this?
I thank You in advance.

Regards
Jorge

Re: How to prevent field names with periods (.) OR hyphens (-) from getting replaced with underscores (_)?

martin_mueller — Sun, 16 Nov 2014 23:00:32 GMT

You could set up your own key-value extraction with CLEAN_KEYS = false... however, that often gets you into trouble when using field names that have non-word characters in them. For example, running ... | eval foo = your-server.de will look for fields called your, server, and de and perform subtraction resp. string concatenation on their values.

I'd say the greater issue here is that you have values used as field names. Your events would be nicer to use if they looked like this: ... server=your-server.de duration=13148. Then you'd have no trouble with cleaned keys and an easy time building reports off the data generically without knowing the domains... which you need to if they're the field names.

Re: How to prevent field names with periods (.) OR hyphens (-) from getting replaced with underscores (_)?

jrodriguezap — Mon, 17 Nov 2014 00:03:22 GMT

Thanks for the tip.
I wish it were as comets, but those who come are dynamic fields, often reaching 10 to 20 "fields =" with different domain names.
consultation, which would put the value CLEAN_KEYS
Currently the SourceType is "report" and am configuring props.conf

[report]
REPORT-rpt_1=no_clean_keys

In transforms.conf

[no_clean_keys]
DELIMS       = ",", "="
CAN_OPTIMIZE = false
MV_ADD       = true
CLEAN_KEYS = false

Extract the fields well, as needed. But keep the above fields, there will be way to clean?

Re: How to prevent field names with periods (.) OR hyphens (-) from getting replaced with underscores (_)?

martin_mueller — Mon, 17 Nov 2014 17:47:58 GMT

Consider going with a structured format such as JSON:

{
  "timestamp": "...",
  "client_host": "...",
  "client_id": "...",
  "report_id": "...",
  "data": [
    {"domain": "...", "duration": 123},
    {"domain": "...", "duration": 456},
    ...
  ]
}

Very easy to parse and work with afterwards, for any number of data points in a single event.

As for your last question, please rephrase - I don't quite grasp what you're asking for.

Re: How to prevent field names with periods (.) OR hyphens (-) from getting replaced with underscores (_)?

jrodriguezap — Mon, 17 Nov 2014 18:07:27 GMT

Hi Martin
What is happening after setting the props and transforms, is showing me the above fields and new fields

client_host  |  client_id  |  report_id  |  igoogleinicio_com  | igoogleinicio.com  |  webair_com |  webair.com  |  smtp_ec_br  | smtp.ec.br  |  your_server_de |  your-server.de

Re: How to prevent field names with periods (.) OR hyphens (-) from getting replaced with underscores (_)?

martin_mueller — Mon, 17 Nov 2014 18:14:22 GMT

The original key-value extractions are still active. Set KV_MODE = none in props.conf to turn them off.

Re: How to prevent field names with periods (.) OR hyphens (-) from getting replaced with underscores (_)?

jrodriguezap — Mon, 17 Nov 2014 18:23:36 GMT

Excellent @martin_mueller!!!
I knew you could
thank you very much

Re: How to prevent field names with periods (.) OR hyphens (-) from getting replaced with underscores (_)?

martin_mueller — Mon, 17 Nov 2014 18:24:59 GMT

I still maintain that using varying domains as field names is going to end in tears...

Re: How to prevent field names with periods (.) OR hyphens (-) from getting replaced with underscores (_)?

jrodriguezap — Mon, 17 Nov 2014 18:28:05 GMT

Martin understand, this is for a report that is generated monthly with a "bucket"
For example: Top 3 mailserver for day
This is stored in a summary, and then it is called by some dashboards.
Thankz

Jorge

Re: How to prevent field names with periods (.) OR hyphens (-) from getting replaced with underscores (_)?

martin_mueller — Mon, 17 Nov 2014 18:29:48 GMT

How is that top 3 mailserver supposed to work if your mailserver domains are the field names?

...oh well, as long as it works for you 🙂