Hi robgora_deloitte, thanks for your effort, could you please guide me on how to fix this issue by applying the customized syntax in props.conf file. ... View more

Hi Robgora, thanks for your effort, we are using the same configuration as given in this add-on, but has I had commented earlier we could notice that for some of the events the dest_port fields are left empty and this due to the values in "service" field with an actual number are parsed as "dest_port" while values in "service" that are resolved as words are not included. So kindly guide me is there a way in splunk configuration which can be applied to overcome this issue. thanks in advance. ... View more

Hi Gurav, the below steps helped me to get the required output. Procedure : 1) First executed a simple search command to filter the sourcetype containing only with the field values called Computer from the seleclted field=subcategory sourcetype=snow:cmdb_ci_list subcategory=Computer 2) Created a Eventtype in the props.conf with the sourcetype=snow:cmdb_ci_list subcategory=Computer along with other exesisting eventtype in the Splunk_Ta_Snow app Eventtype [snow_cmdb_ci_Computer] search = sourcetype=snow:cmdb_ci_list subcategory=Computer 3) Created a Savedsearch query with the newly created eventtype to filter the events contains anything with subcategory of "Computer". [ServiceNow CMDB CI SUB COMP List] disabled = 0 action.email.reportServerEnabled = 0 action.email.useNSSubject = 1 alert.track = 0 cron_schedule = 0 * * * * description = Saved search which populates the CMDB CI contains anything with subcategory of "Computer" dispatch.earliest_time = 0 dispatch.latest_time = now display.general.type = statistics display.visualizations.show = 0 enableSched = 1 request.ui_dispatch_app = search request.ui_dispatch_view = search search = eventtype=snow_cmdb_ci_Computer | dedup sys_id | fields - _bkt, _cd,_indextime,_kv,_raw,_serial,_si,_sourcetype,_subsecond, punct, index, source, sourcetype | inputlookup append=t cmdb_ci_list_lookup | dedup sys_id | outputlookup cmdb_ci_list_lookup 4) After finishing with the above steps the app was pushed to the search head cluster environement via deployer. /opt/splunk/bin ./splunk apply shcluster-bundle --answer-yes -target https://splunkinstancename:8089 -auth admin:password 5) We are able to see the required output in splunk with the events containing the field values called Computer from the seleclted field=subcategory. ... View more

I want to use existing in the query ... View more

Hi Gurav, thanks for your inputs, but I had gone through this document and created the below steps, could please guide me whether steps are correct . 1) Create a empty csv file under Splunk_TA_snow/lookup/cmdb_ci_computer.csv 2) Create a Eventtype [snow_cmdb_ci_Computer] search = sourcetype=snow:cmdb_ci_Computer 3) Map the eventtype in the /local/savedsearches.conf [ServiceNow CMDB CI Computer] disabled = 0 action.email.reportServerEnabled = 0 action.email.useNSSubject = 1 alert.track = 0 cron_schedule = 0 * * * * description = Saved search which populates the CMDB CI Computer from ServiceNow dispatch.earliest_time = 0 dispatch.latest_time = now display.general.type = statistics display.visualizations.show = 0 enableSched = 1 request.ui_dispatch_app = search request.ui_dispatch_view = search search = eventtype=snow_cmdb_ci_computer | dedup sys_id | fields - _bkt, _cd,_indextime,_kv,_raw,_serial,_si,_sourcetype,_subsecond, punct, index, source, sourcetype | inputlookup append=t cmdb_ci_computer_lookup | dedup sys_id | outputlookup cmdb_ci_computer_lookup ... View more

Hi All, Can anyone throw me some lights on this, I want to know to how to configure a lookup in splunk_TA_servicenow. ... View more

Hi All, Any help will be much appreciated. ... View more

Hi All, Can anyone guide me on this ... View more

Hi Frank, I am extremely sorry about this, yes I can understand the pain of supporting based on some information. Are you getting different results between the transforms and the search time rex command for the exact same events? Yes I am getting different output for the same events, between the transforms and the search time rex. Will it be better to share the complete props/transforms.conf in the comment session. so that it will give you a better picture how other stanzas are configured. ... View more

Hi Frank, it worked after changing the name of the stanza, but now got in to another problem the regex is not working as expected , host field value is getting over written but not with the dvc_host value instead its taking the value like ping,dns,ssh,not,splunk etc.. Props.conf: [paloalto:network:traffic] TRANSFORMS-host_override = hostname_override Transforms.conf: [hostname_override] REGEX = (?:[^,]*,){52}(?<host>\w+) DEST_KEY = MetaData:Host FORMAT = host::$1 But the same regex when executed in the search portal getting the required output. index=firewall sourcetype="paloalto:network:traffic" |rex "(?:[^,]*,){52}(?\w+)" | table host dvc_host Kindly guide me on this, not sure where is the problem now. ... View more

No, After testing the opsec, I had removed the opsec related configuration from the test environment. And also un-installed the splunk application and re-installed it again. Let me try a different name for this Paloalto. ... View more

Hi Frank, I had used same set of name for Paloalto app, I mean I had used same stanza which was used in opsec, but mapped to the source type belong to Paloalto Firewall and also changed the regex as per the app. The same set of props.conf/transforms.conf was used in the test and it worked. Props.conf: [paloalto:network:traffic] TRANSFORMS-host_override = host_override [paloalto:network:system] TRANSFORMS-host_override = host_override1 Transforms.conf: [host_override] REGEX = (?:[^,]*,){52}(?<host>\w+) DEST_KEY = MetaData:Host FORMAT = host::$1 [host_override1] REGEX = (?<host>(\w+)$) DEST_KEY = MetaData:Host FORMAT = host::$1 Hey in search head do we need to configure different set of props/transforms to perform search time extraction ? Kindly guide me on this.. ... View more

Hi Frank, Yes Splunk see the data first in the HF servers, then the data are getting indexed in the indexer instances. Environment details: We have 5 heavy forwarder instances, 5 indexer instances, 3 search head cluster members, Deployment server and Deployer server. All the UF forwards the data to the HF and it get parsed and ingested into indexer instances. All the traffic is load balanced by F5 device before entering the splunk instances. We are using Splunk 6.6.1 version both in prod/test. Hey I had copy/pasted complete Paloalto-TA-app details in to test environment and also copy pasted some events from prod to test by going to splunk-->settings-->Add data-->uploaded. And data's where mapped as per the sourcetype configured in the prod environment. I had validated by executing the query without adding the host_over ride stanza first and it did not over ride the host value with the dvc_host field.Then again included the host_over ride stanza and restarted the splunk service and added new set of events data and i could see the host value was getting over ride. Just now I have checked complete TA details in both HF /Search head instances all have same set of configuration. Not sure where is the problem now, could you please guide me on this. ... View more

Hi Frank, coming to this question, I had tested below props.conf / transforms.conf and it worked well in my test environment. Both test/Prod have the same set of props/transforms.conf files but only difference is that test is a single machine, where as prod is a distributed machine. Question : 1) Why the same Props/transforms.conf is not working in prod? In Prod TA-Paloalto is configured in HF instances and search head cluster member. 2) Do I need copy/paste the below stanza details in both Heavy forwarder/search head cluster member? Props.conf: [paloalto:network:traffic] TRANSFORMS-host_override = host_override [paloalto:network:system] TRANSFORMS-host_override = host_override1 Transforms.conf: [host_override] REGEX = (?:[^,]*,){52}(?<host>\w+) DEST_KEY = MetaData:Host FORMAT = host::$1 [host_override1] REGEX = (?<host>(\w+)$) DEST_KEY = MetaData:Host FORMAT = host::$1 Kindly guide me on this. ... View more

Hi Frank, I had copy paste complete event details and when I execute the query for a month period, I could see there are some events have host value as 0,21,test01.xxx.com and Azure in the selected fields. In some events the host field value is located at different position, but in this case is there any away we can remove these host being displayed in host field. Query Detail: index=firewall sourcetype="paloalto:network:log" | rex "(?:[^,]*,){14}(?\w*)" | search host=0 Events Detail: When searched with the host value=0 2/19/18 6:48:32.000 AM 0.0-10.x.x.x,0,2,0,aged-out,50,0,0,0,Ogden-FW,node01,from-policy,,,0,,0,,N/A eventtype = nix-all-logs eventtype = pan network host = 0 source = /opt/syslogs/paloalto/test01.xxx.com/paloalto.log sourcetype = paloalto:network:log tag = network 2/7/18 7:31:22.000 PM Feb 8 00:31:22 test02.xxx.com 1,2018/02/08 00:31:00,012501001041,CORRELATION,,,2018/02/08 00:31:00,168.133.221.8,,,compromised-host,medium,17,0,0,0,,test02,2003071744,Beacon Detection,6005,"Host has made use of Internet Relay Chat (IRC), a protocol popular with command-and-control activity." eventtype = nix-all-logs eventtype = pan network host = 0 source = /opt/syslogs/paloalto/test02.xxx.com/paloalto.log sourcetype = paloalto:network:log tag = network Query Detail: index=firewall sourcetype="paloalto:network:log" | rex "(?:[^,]*,){14}(?\w*)" | search host=test01.xxx.com Event Details: When searched with the host value=test01.xxx.com Feb 4 05:31:09 test01.xxx.com 1,2018/02/04 05:31:09,007257000034869,T eventtype = nix-all-logs eventtype = pan network host = test01.xxx.com source = /opt/syslogs/paloalto/test01.xxx.com/paloalto.log sourcetype = paloalto:network:log tag = network 1/25/18 7:48:05.000 AM om-policy,,,0,,0,,N/A eventtype = nix-all-logs eventtype = pan network host = test01.xxx.com source = /opt/syslogs/paloalto/test01.xxx.com/paloalto.log sourcetype = paloalto:network:log tag = network Query Detail: index=firewall sourcetype="paloalto:network:log" | rex "(?:[^,]*,){14}(?<host>\w*)" | search host=21 Event details: When searched with the host value=21 42643087,1,60564,9080,0,0,0x100053,tcp,allow,4442,1860,2582,23,2018/02/19 13:25:01,21,not-resolved,0,6477528057962830262,0x8000000000000000,10.x.x.x-10.x.x.x,10.x.x.x-10.x.x.x,0,12,11,tcp-rst-from-client,17,0,0,0,Data_Center,test01fw,from-policy,,,0,,0,,N/A eventtype = nix-all-logs eventtype = pan network host = 21 source = /opt/syslogs/paloalto/test01.xxxx.com/paloalto.log sourcetype = paloalto:network:log tag = network Query Detail: index=firewall sourcetype="paloalto:network:log" | rex "(?:[^,]*,){14}(?\w*)" | search host=Azure Event Details: When searched with the host value=Azure RAFFIC,end,1,2018/02/04 05:30:58,10.134.64.7,168.133.4.232,0.0.0.0,0.0.0.0,trust-xxxx,,,dns,vsys1,Azure-Private,Azure-Data-Center,ethernet1/2,ethernet1/1,pan_log_forward,2018/02/04 05:30:58,126039,1,46728,53,0,0,0x4064,udp,allow,x7,x2,x,x,2018/02/04 05:30:28,0,any,0,131004235,0x8000000000000000,10.x.x.x-10.x.x.x,United States,0,1,1,aged-out,115,0,0,0,,node02fw,from-policy,,,0,,0,,N/A Hey I had noticed that we are using 3.8.0 version of Palo alto add-on but I am not sure why the source type was made different in our organisation. I can create another new question in the answer.com for palo alto sourcetype related query. ... View more

Hi Frank, I had used \w* instead of \w+ but still I am able to get the host name as 0, 21,test01.xxx.com and Azure iin the selected host field. when I execute the query for a duration of 1 month. Kindly guide how we can correct this, as in the selected host field we are able to see the host information as 0,21,Azure,test01.xxx.com and a blank space. ... View more

Hi Douglashurd, Source fire issue got fixed with out upgrading the source fire app. Initially we had upgraded the app but it did not work, so removed the newly upgraded app from splunk heavy forwarder instances. We fixed the issue by executing the below command from the /opt/splunk/etc/apps/estreamerapp/bin ./estreamer_client.pl -c config_nogui.sh on executing the above command, we were able to search the logs via splunk console. ... View more

Hi Frank, Do I need to update the props/transforms in search head instances also, because I could see same configuration in search head cluster member. ... View more

Hi Frank, in some case the host_name is not there in event logs in that case the above regex fetches the actual host name "test01.xxx.com" in the host field. When we searched with the below query for 24 hours duration, we could see this events in which there is no host name available. In this case what can be done, to remove the actual host from the host field. index=firewall sourcetype="network:log" | rex "(?:[^,]*,){14}(?\w+)" Example : Feb 19 13:25:33 test01.xxx.com 1,2018/02/19 13:25:33,012501001041,TRAFFIC,end,1,2018/02/19 13:25:24,10.x.x.x,10.x.x.x,0.0.0.0,0.0.0.0,xxxx_not_here,,,ssl,vsys2,Data-Center-Admin,Data-Center-Core,ae5.2005,ae5.250,pan_log_forward,2018/02/19 13:25:24, ... View more

Hi Frank, Hey the same regex which was used to over right the host values with the interesting field values, all the regex worked fine in the search portal. But when the same was updated in the props.conf and transforms.conf it did not work. We have props/ transforms.conf in Heavy forwarder instances. Props.conf details: [paloalto:network:traffic] TRANSFORMS-host_override = host_override [paloalto:network:tsys] TRANSFORMS-host_override = host_override1 [paloalto:network:log] TRANSFORMS-host_override = host_override2 Transforms.conf Details: [host_override] REGEX =(?<host>\b[^(\.),]+\b(?=,(?=from-policy|from-application))\b) DEST_KEY = MetaData:Host FORMAT = host::$1 [host_override1] REGEX =(?<host>(\w+)$) DEST_KEY = MetaData:Host FORMAT = host::$1 [host_override2] REGEX =(?:[^,]*,){14}(?<host>\w+) DEST_KEY = MetaData:Host FORMAT = host::$1 Kindly guide me on this. ... View more

Hi Frank, the above regex worked and we are able to see the host_name field value in the host value of the selected field. index=firewall sourcetype="network:log" | rex "(?:[^,]*,){14}(?\w+)" thanks for the much needed help. ... View more

Hi Frank, Yes you are right not all the events are preceding with the "Client_VPN" so that events which are not preceding the value of Client_VPN are throwing the actual host name "test01.xxx.com" in the host field of selected field. Then I had tried to locate the position of the host_name field in the events but unfortunately I am unable to narrow down the exact position. So instead of matching the Client_VPN, most of the host name comes before this value ",3,vsys ". Used the below Regex | rex field=_raw "(?\b[^(\.),]+\b(?=,\d,vsys*)\b)" Feb 16 23:53:54 test01.xxxx.com 1,2018/02/16 23:53:54,012501001041,6477528057870932374,0x8000000000000000,USERID,logout,369,2018/02/16 23:53:47,36,0,0,0,Client_VPN,node01fw,3,vsys3,10.X.X.X,ddesa0002,,0,1,0,0,0,vpn-client,globalprotect,0,0,,2018/02/16 23:53:47,1 Due to this the regex was unable to over right the host value with the host_name field value. Kindly guide in the regex to over write the host value with the host_name. ... View more

Hi Niketnilay, Yes it is the part the previous answer, but for the different scenario, where the host name comes in between the events and the value of the host field which needs to over written with the host_name. I had tried for regex but it did not work, so could please guide me on the regex. ... View more