How to extract fields from csv file?

woodentree — Tue, 25 Feb 2020 10:46:02 GMT

Hello,

We use a python script to export some data every 24 hours from our database and save it in $SPLUNK_HOME/etc/apps/SplunkEnterpriseSecuritySuite/lookups folder in .csv format. For some reason Splunk can't recognise ; as a delimiter, so we have a lookup with a single field like below:

name;ip;os;environment
wks123;192.168.0.1;windows 10;production
srv456;192.168.0.2;widows 2016;test
etc.

At the same time when we create a new lookup file based on the same .csv file via Lookup Editor add-on it works perfectly fine.

Could you please help as to set up a delimiter for our original .csv file in Splunk configuration?

Thanks.

Re: How to extract fields from csv file?

to4kawa — Tue, 25 Feb 2020 13:30:54 GMT

transforms.conf

DELIMS = <quoted string list>
* NOTE: This setting is only valid for search-time field extractions.
* IMPORTANT: If a value may contain an embedded unescaped double quote
  character, such as "foo"bar", use REGEX, not DELIMS. An escaped double
  quote (\") is ok. Non-ASCII delimiters also require the use of REGEX.
* Optional. Use DELIMS in place of REGEX when you are working with ASCII-only
  delimiter-based field extractions, where field values (or field/value pairs)
  are separated by delimiters such as colons, spaces, line breaks, and so on.
* Sets delimiter characters, first to separate data into field/value pairs,
  and then to separate field from value.
* Each individual ASCII character in the delimiter string is used as a
  delimiter to split the event.
* Delimiters must be specified within double quotes (eg. DELIMS="|,;").
  Special escape sequences are \t (tab), \n (newline), \r (carriage return),
  \\ (backslash) and \" (double quotes).
* When the event contains full delimiter-separated field/value pairs, you
  enter two sets of quoted characters for DELIMS:
* The first set of quoted delimiters extracts the field/value pairs.
* The second set of quoted delimiters separates the field name from its
  corresponding value.
* When the event only contains delimiter-separated values (no field names),
  use just one set of quoted delimiters to separate the field values. Then use
  the FIELDS setting to apply field names to the extracted values.
  * Alternately, Splunk software reads even tokens as field names and odd
    tokens as field values.
* Splunk software consumes consecutive delimiter characters unless you
  specify a list of field names.
* The following example of DELIMS usage applies to an event where
  field/value pairs are separated by '|' symbols and the field names are
  separated from their corresponding values by '=' symbols:
    [pipe_eq]
    DELIMS = "|", "="
* Default: ""

Re: How to extract fields from csv file?

woodentree — Tue, 25 Feb 2020 15:30:51 GMT

Thanks for the help, but it still seems unclear for me. Could you please provide an example or develop your answer.
Many thanks.

topic Re: How to extract fields from csv file? in Splunk Enterprise Security

How to extract fields from csv file?

Re: How to extract fields from csv file?

Re: How to extract fields from csv file?