Dashboards & Visualizations

.

svenrieder
Explorer

This Question got deleted.

Labels (1)
0 Karma
1 Solution

gcusello
Legend

Hi @svenrieder,

I found that sometimes some of the fields could be empty.

This can be managed using "*" instead "+" in the regex, something like this:

index=your_index
| rex field=Data "\"(?<Plugin_Name>[^\"]*)\",\"(?<Severity>[^\"]*)\",\"(?<IP_Address>]\"]*)\",\"(?<NetBIOS_Name>[^\"]*)\",\"(?<DNS_Name>[^\"]*)\",\"(?<MAC_Address>[^\"]*)\",\"(?<Repository>[^\"]*)\",\"(?<Agent_ID>[\"]*)\""
| table PluginName Severity IP_Address NetBIOS_Name DNS_Name MAC_Address Repository Agent_ID

Ciao.

Giuseppe

View solution in original post

gcusello
Legend

Hi @svenrieder,

you have to index your csv using the INDEXED_EXTRACTION mode, for more infos see at https://docs.splunk.com/Documentation/Splunk/8.2.4/Data/Extractfieldsfromfileswithstructureddata

in few words, you have to put in your props.conf:

[your_sourcetype]
HEADER_FIELD_LINE_NUMBER=1
TIMESTAMP_FIELDS=TimeStamp
FIELD_DELIMITER=,
FIELD_QUOTE="

I don't see in your sample the TimeStamp, if you haven't don't use the TIMESTAMP_FIELDS option.

The easiest approach is to try to index a sample of your csv files using the "Add Data" function [Settings -- Add Data], that guides you in the sourcetype definition.

Ciao.

Giuseppe

0 Karma

svenrieder
Explorer

Hi @gcusello,

I can't do that, as I work in a large Company and dont have the rights to do that.

Isn't it possible to do that with a Splunk Search? If not it would also be possible for me to write every row into a seperate log-file if its possible that way.

0 Karma

PickleRick
Ultra Champion

Ask "the splunk guys" to define proper extractions. After all that's the point of using splunk - to have actionable data. If they're just ingesting everything "as is" without caring what it's used for, you're kind of wasting your license.

And you can define your own field extractions so you don't have to type in the rex command every time.

0 Karma

gcusello
Legend

Hi @svenrieder,

yes, you could extract all the fields at search time

index=your_index
| rex "\"(?<Plugin_Name>[^\"]+)\",\"(?<Severity>[^\"]+)\",\"(?<IP_Address>]\"]+)\",\"(?<NetBIOS_Name>[^\"]+)\",\"(?<DNS_Name>[^\"]+)\",\"(?<MAC_Address>[^\"]+)\",\"(?<Repository>[^\"]+)\",\"(?<Agent_ID>[\"]+)\""
| table PluginName Severity IP_Address NetBIOS_Name DNS_Name MAC_Address Repository Agent_ID

Ciao.

Giuseppe

svenrieder
Explorer

Hi @gcusello 

Thats perfect thank you! Now the string with that Information is in the Field "Data" From the Event:

svenrieder_1-1644259184092.png

How do you tell the rex where the information to filter is?

0 Karma

gcusello
Legend

Hi @svenrieder,

as you can see at https://docs.splunk.com/Documentation/Splunk/8.2.4/SearchReference/Rex 

you have to little modify the search:

index=your_index
| rex field=Data "\"(?<Plugin_Name>[^\"]+)\",\"(?<Severity>[^\"]+)\",\"(?<IP_Address>]\"]+)\",\"(?<NetBIOS_Name>[^\"]+)\",\"(?<DNS_Name>[^\"]+)\",\"(?<MAC_Address>[^\"]+)\",\"(?<Repository>[^\"]+)\",\"(?<Agent_ID>[\"]+)\""
| table PluginName Severity IP_Address NetBIOS_Name DNS_Name MAC_Address Repository Agent_ID

 addinf "field=Data" to the rex command.

Ciao.

Giuseppe

svenrieder
Explorer

Hi @gcusello 

Somehow it is still not working. It is not showing the results in the table:

svenrieder_0-1644400086599.png

 

0 Karma

gcusello
Legend

Hi @svenrieder,

this means that somehow there's some difference in the log format, you have to identify these differences and modify the regex used for the extractions.

If you could identify the events with different logs and share them I could help you in regex update.

Ciao.

Giuseppe

0 Karma

svenrieder
Explorer

Hi @gcusello 

Thanks for your Help! The Event looks like this:

svenrieder_0-1644404381181.png

 

0 Karma

gcusello
Legend

Hi @svenrieder,

let me understand: in each event you have the header (that contains all the field names) and the values or only on some events?

my regex is done for events with only the values, without the field names.

if you have some events with only the values and others with header and values, I hint to skeak with the people who created the parser because there's an error in parsing!

Anyway, in this case you could use two different regexes: one for events with only values (the one I already sent you) and another one for the events with both field names and values:

index=your_index
| rex field=Data "\"(?<Plugin_Name>[^\"]*)\",\"(?<Severity>[^\"]*)\",\"(?<IP_Address>]\"]*)\",\"(?<NetBIOS_Name>[^\"]*)\",\"(?<DNS_Name>[^\"]*)\",\"(?<MAC_Address>[^\"]*)\",\"(?<Repository>[^\"]*)\",\"(?<Agent_ID>[\"]*)\""
| rex field=Data "^(\"[^\"]*\",){7}\"[^\"]*\"\s+\"(?<Plugin_Name>[^\"]*)\",\"(?<Severity>[^\"]*)\",\"(?<IP_Address>[^\"]*)\",\"(?<Netbios_Name>[^\"]*)\",\"(?<DNS_Name>[^\"]*)\",\"(?<MAC_Address>[^\"]*)\",\"(?<Repository>[^\"]*)\",\"(?<Agent_ID>[^\"]*)\""
| table PluginName Severity IP_Address NetBIOS_Name DNS_Name MAC_Address Repository Agent_ID

 Ciao.

Giuseppe

0 Karma

svenrieder
Explorer

Hi @gcusello 

I can delete the top line out before I send it to Splunk, as the Search is easier to understand and looks nicer.
So in the Field Data I only have the Values without the Header.

My Question is, this
| rex field=Data "\"(?<Name>[^\"]*)\",\"(?<Website>[^\"]*)\",\"(?<Country>[^\"]*)\""
Only reads out the first line.

The thing is, that when I write the values from the csv-File into a Log-File, it adds all rows together so if it was:

Line 1
Line 2
Line 3

In the Log it will be:
Line 1 Line 2 Line 3

So if I have this in the CSV
"Thomas","thomas.com","England"
"Luca","luca.com","USA"
In the Log-File and therefore also in the Field of the event it looks like this:
"Thomas","thomas.com","England" "Luca","luca.com","USA"

Now my Question was, if it is possible to make a table with only one Event, which has all Lines in it and if yes, how do the Lines have to be Formatted in the Log and how does the rex look like to show it like this:

svenrieder_0-1644417652695.png

(Here I made 2 Events, for every line a sepearate Event)

 

0 Karma

gcusello
Legend

Hi @svenrieder,

I found that sometimes some of the fields could be empty.

This can be managed using "*" instead "+" in the regex, something like this:

index=your_index
| rex field=Data "\"(?<Plugin_Name>[^\"]*)\",\"(?<Severity>[^\"]*)\",\"(?<IP_Address>]\"]*)\",\"(?<NetBIOS_Name>[^\"]*)\",\"(?<DNS_Name>[^\"]*)\",\"(?<MAC_Address>[^\"]*)\",\"(?<Repository>[^\"]*)\",\"(?<Agent_ID>[\"]*)\""
| table PluginName Severity IP_Address NetBIOS_Name DNS_Name MAC_Address Repository Agent_ID

Ciao.

Giuseppe

svenrieder
Explorer

Hi @gcusello 

Okay  it worked, thanks! I now used different attributes for testing the Search:
Name, Website, Country

The log is read by Splunk without making a new line which means the csv that looked like this:
"Thomas","thomas.com","England"
"Luca","luca.com","USA"

Looks like this in Splunk:
"Thomas","thomas.com","England" "Luca","luca.com","USA"

So the Search is showing the Table with the first line only:

svenrieder_0-1644415347076.png

How do I have to configure the log input or the splunk search so it lists every line?

 

0 Karma
Get Updates on the Splunk Community!

Improve Your Security Posture

Watch NowImprove Your Security PostureCustomers are at the center of everything we do at Splunk and security ...

Maximize the Value from Microsoft Defender with Splunk

 Watch NowJoin Splunk and Sens Consulting for this Security Edition Tech TalkWho should attend:  Security ...

This Week's Community Digest - Splunk Community Happenings [6.27.22]

Get the latest news and updates from the Splunk Community here! News From Splunk Answers ✍️ Splunk Answers is ...