Streaming app - netflow - data not parsed

[heskez]

Hi There, I've netflow forwarding configured and streaming app installed.
I'm receiving the netflow data:

sourcetype="stream.netflow"

3/24/18
1:04:26.190 PM
{ [-]
app:
bytes: 0
count: 1
dest_ip: 8.8.8.8
dest_port: 53
drop_packet_count: 0
endtime: 2018-03-24T12:04:26.190502Z
packets: 0
packets_in: 1
packets_out: 1
src_ip: 192.168.1.2
src_mac:
src_port: 54808
sum(bytes_in): 73
sum(bytes_out): 73
timestamp: 2018-03-24T12:04:26.190502Z
}
Show as raw text

However, when i click the Stream application nothing is shown in the dashboard, analytics overview or flow visualization. -> No results found. Looks like the data isn't parsed by Splunk Streaming app.

[heskez]

This mystery is still not solved.. Yes, I get data now with this query index= source="stream:fortistream" in verbose mode:

But still ip addresses are only showing the first two octets..

Hmm, I suspect the regex: rex field=src_ip "(?<src1>.*)\.(?<src2>.*)\.(?<src3>.*)\.(?<src4>.*)"

basically the field ip is calculated as: (?:(?<!\d)(\d{1,3}\.\d{1,3}\.\d{1,3}\.\d{1,3})(?!\d))

So where is the syntax error...????

Anyone?

[heskez]

So next I've created a new stream "fortistream" protocol netflow.
Then I've edited the search from the VF dashboard like this:

index= source="stream:fortistream" | rex field=src_ip "(?.).(?.).(?.).(?.)" | where src1 NOT null | rex field=dest_ip "(?.).(?.).(?.).(?.)" | where dest1 NOT null | eval source_ip=round(src1+exact(src2.001), 3) | eval destination_ip=round(dest1+exact(dest2*.001), 3) | eventstats sum(sum(bytes)) as bytes by source_ip, destination_ip | stats latest(source_ip), latest(destination_ip), sum(count) by bytes | rename latest(source_ip) as "Source IP", latest(destination_ip) as "Destination IP", sum(count) as "Flows", bytes as "Bytes", sourcetype as "Sourcetype"

Put it in verbose mode.

Now I do get data but ONLY the flow data. And here is the funny thing:
The source ip address is cut in half.. (192.168)
The destination ip address too (8.008)

Very weird..

[heskez]

When I run the search like that "index=* source=stream:Splunk_IP" I don't get any results..

[Azeemering]

I think you need to look at the SPL behind the dashboards and see if it matches the sourcetype that you have.
For example that specific dashboard you mention (Flow Visualization) has the following spl:

index=* source=stream:Splunk_IP | rex field=src_ip "(?.).(?.).(?.).(?.)" | where src1 NOT null | rex field=dest_ip "(?.).(?.).(?.).(?.)" | where dest1 NOT null | eval source_ip=round(src1+exact(src2*.001), 3) | eval destination_ip=round(dest1+exact(dest2*.001), 3) | eventstats sum(sum(bytes)) as bytes by source_ip, destination_ip | stats latest(source_ip), latest(destination_ip), sum(count) by bytes | rename latest(source_ip) as "Source IP", latest(destination_ip) as "Destination IP", sum(count) as "Flows", bytes as "Bytes", sourcetype as "Sourcetype"

So take off all spl from that and search only index=* source=stream:Splunk_IP. Do you get any data in verbose mode?

[heskez]

I do get data running this query btw:
index=* source="stream:*" splunk_server="host_ip"

[heskez]

This query doesn't return data (This is extracted from the SPL):
index=* source="stream:Splunk_Tcp"

[heskez]

This query doesn't return data either (Extracted from flow visualisation)
index=* source=stream:Splunk_IP

[heskez]

So next I've created a new stream "fortistream" protocol netflow.
Then I've edited the search from the VF dashboard like this:

index=* source="stream:fortistream" | rex field=src_ip "(?.).(?.).(?.).(?.)" | where src1 NOT null | rex field=dest_ip "(?.).(?.).(?.).(?.)" | where dest1 NOT null | eval source_ip=round(src1+exact(src2*.001), 3) | eval destination_ip=round(dest1+exact(dest2*.001), 3) | eventstats sum(sum(bytes)) as bytes by source_ip, destination_ip | stats latest(source_ip), latest(destination_ip), sum(count) by bytes | rename latest(source_ip) as "Source IP", latest(destination_ip) as "Destination IP", sum(count) as "Flows", bytes as "Bytes", sourcetype as "Sourcetype"

Put it in verbose mode.

Now I do get data but ONLY the flow data. And here is the funny thing:
The source ip address is cut in half.. (192.168)
The destination ip address too (8.008)

Very weird..