topic Re: Testing - uploaded Check Point files and granularity in Knowledge Management

Testing - uploaded Check Point files and granularity

tdthorwald — Tue, 30 Apr 2019 08:39:33 GMT

hello,

I am currently testing Splunk, with a single instance on a VM.
I have some trouble getting information out of logs correctly.
The log I am analysing has the following fields:
Time Stamp, Action, Source, Destination, Translated, Source, Translated Dest, Duration, Bytes Sent, Bytes Received, Application, and Reason.
some sample data:

========================================================================================================================
Entire Traffic Log list
Current system time is Thu, 25 Apr 2019 09:38:19
========================================================================================================================

Time Stamp          Action  Source     Destination           Translated Source     Translated Dest       Duration     Bytes Sent Bytes Received Application Reason                    


2019-04-25 09:38:19 Permit 
10.11.100.139:49573   192.168.3.2:9090      10.11.100.139:49573   192.168.3.2:9090      0 sec                 0              0 TCP PORT 9090 Creation                 2019-04-25 09:38:19 Permit 
10.11.100.104:52934   <public IP>:443     <public IP>:30233   <public IP>:443    0 sec                 0              0 HTTPS       Creation                   2019-04-25 09:38:19 Deny   
10.10.1.50:60239      <public IP>:443     0.0.0.0:0             0.0.0.0:0             0 sec                 0             28 HTTPS       Traffic Denied             2019-04-25 09:38:19 Permit 
10.11.100.139:49572   192.168.3.2:9090      10.11.100.139:49572   192.168.3.2:9090      0 sec                 0              0 TCP PORT 9090 Creation                 2019-04-25 09:38:19 Permit 
10.11.100.133:50622   <public IP>:443     <public IP>:32209   <public IP>:443    0 sec                 0              0 HTTPS       Creation                   2019-04-25 09:38:19 Permit 
10.11.100.139:49571   192.168.3.2:9090      10.11.100.139:49571   192.168.3.2:9090      0 sec                 0              0 TCP PORT 9090 Creation                 2019-04-25 09:38:19 Permit 
10.11.100.39:51561    <public IP>:443     <public IP>:57732   <public IP>:443    0 sec                 0              0 HTTPS

That's the first few lines of the log.
I have replaced public IPs with <public IP> for obvious reasons.

When I try to transform all these so I can select on them more easily, I run into errors.

What is the best way to get the data out?

I guess I have to change a props.conf file. How do I find the one that contains the sourcetype I created?

Re: Testing - uploaded Check Point files and granularity

koshyk — Fri, 03 May 2019 21:22:57 GMT

your question/problem seems to be very generic. would be good to put the actual event message etc.

So at which point you stuck?
1. Are you able to index data into Splunk? Check if inputs.conf is correct
2. Did you specify the indextime settings correctly? (ie. timestamp, source, host, sourcetype, line break etc.) all within props.conf
3. Once (1) and (2) is complete, ensure you extract all basic things like sourcetype, time etc.

Re: Testing - uploaded Check Point files and granularity

tdthorwald — Mon, 06 May 2019 07:52:39 GMT

Hello Koshyk,

1) The data goes into Splunk fine.
2) Yes. the line break is fine.

Even Sourcetype, time, source are extracted correctly. But an event is just that, an event. I cannot select on sourceIP, or protocol.
I can select a sourceIP from any event, but not all SourceIPs, because Splunk does not see them as key/value pairs. It only sees the default key/value pairs.

My question is: where do I define them? in the inputs.conf file?
If I look for "inputs.conf" I get 26 hits (the VM is both Indexer, UF ánd SH...)
In SPLUNK\etc\system\default** the sourcetype I configured does not appear in either **inputs.conf or props.conf.

Re: Testing - uploaded Check Point files and granularity

koshyk — Mon, 06 May 2019 07:57:25 GMT

you need to provide sample events and we can write the props.conf for you

these settings are normally in props.conf with sometimes, (for complex extractions) transforms.conf

Re: Testing - uploaded Check Point files and granularity

tdthorwald — Wed, 08 May 2019 07:22:39 GMT

I changed the question to include some sample events.

Re: Testing - uploaded Check Point files and granularity

tdthorwald — Tue, 28 May 2019 13:46:01 GMT

I updated the question with an example...
Can anyone help me?