Splunk Search

How to prevent field names with periods (.) OR hyphens (-) from getting replaced with underscores (_)?

jrodriguezap
Communicator

Hi
I have the following logs:

10/01/2014 00:00:00 -0500, client_host="172.24.1.41", client_id=db01, report_id=RAS04, igoogleinicio.com=3491,  webair.com=13148
10/01/2014 00:00:00 -0500, client_host="172.24.1.41", client_id=db01, report_id=RAS04, smtp.ec.pe=1313,  your-server.de=13148

These data need to view them as a table: ... | table *
And show me the header like this:

client_host  |  client_id  |  report_id  |  igoogleinicio_com  |  webair_com  |  smtp_ec_br  |  your_server_de

As you can see, the fields that have: _ and . are replaced by _ (underline).
There will be no way around this?
I thank You in advance.

Regards
Jorge

Tags (3)
0 Karma
1 Solution

martin_mueller
SplunkTrust
SplunkTrust

You could set up your own key-value extraction with CLEAN_KEYS = false... however, that often gets you into trouble when using field names that have non-word characters in them. For example, running ... | eval foo = your-server.de will look for fields called your, server, and de and perform subtraction resp. string concatenation on their values.

I'd say the greater issue here is that you have values used as field names. Your events would be nicer to use if they looked like this: ... server=your-server.de duration=13148. Then you'd have no trouble with cleaned keys and an easy time building reports off the data generically without knowing the domains... which you need to if they're the field names.

View solution in original post

martin_mueller
SplunkTrust
SplunkTrust

You could set up your own key-value extraction with CLEAN_KEYS = false... however, that often gets you into trouble when using field names that have non-word characters in them. For example, running ... | eval foo = your-server.de will look for fields called your, server, and de and perform subtraction resp. string concatenation on their values.

I'd say the greater issue here is that you have values used as field names. Your events would be nicer to use if they looked like this: ... server=your-server.de duration=13148. Then you'd have no trouble with cleaned keys and an easy time building reports off the data generically without knowing the domains... which you need to if they're the field names.

View solution in original post

martin_mueller
SplunkTrust
SplunkTrust

How is that top 3 mailserver supposed to work if your mailserver domains are the field names?

...oh well, as long as it works for you 🙂

0 Karma

martin_mueller
SplunkTrust
SplunkTrust

I still maintain that using varying domains as field names is going to end in tears...

0 Karma

jrodriguezap
Communicator

Martin understand, this is for a report that is generated monthly with a "bucket"
For example: Top 3 mailserver for day
This is stored in a summary, and then it is called by some dashboards.
Thankz

Jorge

0 Karma

martin_mueller
SplunkTrust
SplunkTrust

The original key-value extractions are still active. Set KV_MODE = none in props.conf to turn them off.

jrodriguezap
Communicator

Excellent @martin_mueller!!!
I knew you could
thank you very much

0 Karma

martin_mueller
SplunkTrust
SplunkTrust

Consider going with a structured format such as JSON:

{
  "timestamp": "...",
  "client_host": "...",
  "client_id": "...",
  "report_id": "...",
  "data": [
    {"domain": "...", "duration": 123},
    {"domain": "...", "duration": 456},
    ...
  ]
}

Very easy to parse and work with afterwards, for any number of data points in a single event.

As for your last question, please rephrase - I don't quite grasp what you're asking for.

0 Karma

jrodriguezap
Communicator

Hi Martin
What is happening after setting the props and transforms, is showing me the above fields and new fields

client_host  |  client_id  |  report_id  |  igoogleinicio_com  | igoogleinicio.com  |  webair_com |  webair.com  |  smtp_ec_br  | smtp.ec.br  |  your_server_de |  your-server.de
0 Karma

jrodriguezap
Communicator

Thanks for the tip.
I wish it were as comets, but those who come are dynamic fields, often reaching 10 to 20 "fields =" with different domain names.
consultation, which would put the value CLEAN_KEYS
Currently the SourceType is "report" and am configuring props.conf

[report]
REPORT-rpt_1=no_clean_keys

In transforms.conf

[no_clean_keys]
DELIMS       = ",", "="
CAN_OPTIMIZE = false
MV_ADD       = true
CLEAN_KEYS = false

Extract the fields well, as needed. But keep the above fields, there will be way to clean?

0 Karma
Did you miss .conf21 Virtual?

Good news! The event's keynotes and many of its breakout sessions are now available online, and still totally FREE!