Oh, fixed it. In my particular case, it was just a permissions issue, as the importtool command runs as "splunk" user,  and I had the .csv files as root with 600 permissions, so, make sure splunk user have read access to the .csv files. So, a command like this should work: /opt/splunk/bin/splunk cmd importtool /[splunk_hot_data_dir]/[indexname]/db/[new.bucket.name] exported.indexname.db.file.csv remember also that you need to specify a new bucket directory inside "db" dir, (you can create it manually or splunk will do it for you)   ... View more

Nice to hear that! Hope this help someone else. ... View more

Hi Everyone, We are using the SEP 14.2.1 (14.2 RU1 MP1) build 4815 (14.2.4815.1101) Installed the last Symantec Add-On, and since the Risk were not correctly tagged, I have modified the regex like this : (you can put it on local/transforms.conf) [field_extraction_for_agt_risk] ### Modified Regex, removed unknown tag that brokes the regex, and moved certificates tags to the end to be recognized. REGEX = (?i)(?:[[sep_file_prefix]]),\s*(?<Risk_Action>[[sep_file_field]]),\s*(?:IP\sAddress:\s*(?<IP_Address>[[sep_file_field]]))?,\s*(?:Computer\sname:\s*(?<Computer_Name>[[sep_file_field]]))?,\s*(?:Source:\s*(?<Source>[[sep_file_field]]))?,\s*(?:Risk\sname:\s*(?<Risk_Name>[[sep_file_field]]))?,\s*(?:Occurrences:\s*(?<Occurrences>[[sep_file_field]]))?,\s*(?<file_path>[[sep_file_field]]),\s*(?<Description>[[sep_file_field]]),\s*(?:Actual\saction:\s*(?<vendor_action>[[sep_file_field]]))?,\s*(?:Requested\saction:\s*(?<Requested_Action>[[sep_file_field]]))?,\s*(?:Secondary\saction:\s*(?<Secondary_Action>[[sep_file_field]]))?,\s*(?:Event\stime:\s*(?<Event_Time>[[sep_file_field]]))?,\s*(?:Inserted:\s*(?<Event_Insert_Time>[[sep_file_field]]))?,\s*(?:End:\s*(?<End_Time>[[sep_file_field]]))?,\s*(?:Last\supdate\stime:\s*(?<Last_Update_Time>[[sep_file_field]]))?,\s*(?:Domain:\s*(?<Domain_Name>[[sep_file_field]]))?,\s*(?:Group:\s*(?<Group_Name>[[sep_file_field]]))?,\s*(?:Server:\s*(?<Server_Name>[[sep_file_field]]))?,\s*(?<user>[[sep_file_field]]),\s*(?:Source\scomputer:\s*(?<Source_Computer_Name>[[sep_file_field]]))?,\s*(?:Source\sIP:\s*(?<Source_Computer_IP>[[sep_file_field]]))?,\s*(?:Disposition:\s*(?<Disposition>[[sep_file_field]]))?,\s*(?:Download\ssite:\s*(?<Download_Site>[[sep_file_field]]))?,\s*(?:Web\sdomain:\s*(?<Web_Domain>.*))?,\s*(?:Downloaded\sby:\s*(?<Downloaded_By>[[sep_file_field]]))?,\s*(?:Prevalence:\s*(?<Prevalence>[[sep_file_field]]))?,\s*(?:Confidence:\s*(?<Confidence>[[sep_file_field]]))?,\s*(?:URL\sTracking\sStatus:\s*(?<URL_Tracking_Status>[[sep_file_field]]))?,\s*(?:First\sseen:\s*(?<First_Seen>[[sep_file_field]]))?,\s*(?:Sensitivity:\s*(?<Sensitivity>[[sep_file_field]]))?,\s*(?<Reason_For_White_Listing>[[sep_file_field]]),\s*(?:Application\shash:\s*(?<Application_Hash>[[sep_file_field]]))?,\s*(?:Hash\stype:\s*(?<Hash_Type>[[sep_file_field]]))?,\s*(?:Company\sname:\s*(?<Company_Name>.*))?,\s*(?:Application\sname:\s(?<Application_Name>[[sep_file_field]]))?,\s*(?:Application\sversion:\s*(?P<Application_Version>.*))?,\s*(?:Application\stype:\s*(?<Application_Type>[[sep_file_field]]))?,\s*(?:File\ssize\s\(bytes\):\s*(?<File_Size>[[sep_file_field]]))?(?:,\s*Category\sset:\s*(?<Category_Set>[[sep_file_field]]),\s*Category\stype:\s*(?<Category_Type>[[sep_file_field]]))?,?\s*(?:Location:\s*(?<Location>[[sep_file_field]]))?,?\s*(?:Intensive\sProtection\sLevel:\s*(?<Intensive_Protection_Level>[[sep_file_field]]))?,?\s*(?:Certificate\sissuer:\s*(?<Certificate_Issuer>[[sep_file_field]]))?,?\s*(?:Certificate\ssigner:\s*(?<Certificate_Signer>[[sep_file_field]]))?,?\s*(?:Certificate\sthumbprint:\s*(?<Certificate_Thumbprint>[[sep_file_field]]))?,?\s*(?:Signing\stimestamp:\s*(?<Signing_Timestamp>[[sep_file_field]]))?,?\s*(?:Certificate\sserial\snumber:\s*(?<Certificate_Serial_Number>[[sep_file_field]]))? [field_extraction_for_agt_proactive] ### Modified Regex, moved certificate tags to the end like the agt_risk one. REGEX = (?i)(?:[[sep_file_prefix]]),\s*(?<Risk_Action>[[sep_file_field]]),\s*(?:Computer\sname:\s*(?<Computer_Name>[[sep_file_field]]))?,?\s*(?:IP\sAddress:\s*(?<IP_Address>[[sep_file_field]]))?,\s*(?:Detection\stype:\s*(?<Detection_Type>[[sep_file_field]]))?,\s*(?:First\sseen:\s*(?<First_Seen>[[sep_file_field]]))?,\s*(?:Application\sname:\s*(?<Application_Name>[[sep_file_field]]))?,\s*(?:Application\stype:\s*(?<Application_Type>[[sep_file_field]]))?,\s*(?:Application\sversion:\s*(?<Application_Version>[[sep_file_field]]))?,\s*(?:Hash\stype:\s*(?<Hash_Type>[[sep_file_field]]))?,\s*(?:Application\shash:\s*(?<Application_Hash>[[sep_file_field]]))?,\s*(?:Company\sname:\s*(?<Company_Name>.*))?,\s*(?:File\ssize\s\(bytes\):\s*(?<File_Size>[[sep_file_field]]))?,\s*(?:Sensitivity:\s*(?<Sensitivity>[[sep_file_field]]))?,\s*(?:Detection\sscore:\s*(?<Detection_Score>[[sep_file_field]]))?,\s*(?:COH\sEngine\sVersion:\s*(?<COH_Engine_Version>[[sep_file_field]]))?,\s*(?<Submission_Recommendation>[[sep_file_field]]),\s*(?:Permitted\sapplication\sreason:\s*(?<Permitted_Application_Reason>[[sep_file_field]]))?,\s*(?:Disposition:\s*(?<Disposition>[[sep_file_field]]))?,\s*(?:Download\ssite:\s*(?<Download_Site>[[sep_file_field]]))?,\s*(?:Web\sdomain:\s*(?<Web_Domain>.*))?,\s*(?:Downloaded\sby:\s*(?<Downloaded_By>[[sep_file_field]]))?,\s*(?:Prevalence:\s*(?<Prevalence>[[sep_file_field]]))?,\s*(?:Confidence:\s*(?<Confidence>[[sep_file_field]]))?,\s*(?:URL\sTracking\sStatus:\s*(?<URL_Tracking_Status>[[sep_file_field]]))?,\s*(?:Risk\sLevel:\s*(?<Risk_Level>[[sep_file_field]]))?,?\s*(?:Risk\stype:\s*(?<Risk_Type>[[sep_file_field]]))?,?\s*(?:Detection\sSource:\s*(?<Detection_Source>[[sep_file_field]]))?,\s*(?:Source:\s*(?<Source>[[sep_file_field]]))?,\s*(?:Risk\sname:\s*(?<Risk_Name>[[sep_file_field]]))?,\s*(?:Occurrences:\s*(?<Occurrences>[[sep_file_field]]))?,\s*(?<file_path>[[sep_file_field]]),\s*(?<Description>[[sep_file_field]]),\s*(?:Actual\saction:\s*(?<vendor_action>[[sep_file_field]]))?,\s*(?:Requested\saction:\s*(?<Requested_Action>[[sep_file_field]]))?,\s*(?:Secondary\saction:\s*(?<Secondary_Action>[[sep_file_field]]))?,\s*(?:Event\stime:\s*(?<Event_Time>[[sep_file_field]]))?,\s*(?:Inserted:\s*(?<Event_Insert_Time>[[sep_file_field]]))?,\s*(?:End:\s*(?<End_Time>[[sep_file_field]]))?,\s*(?:Domain:\s*(?<Domain_Name>[[sep_file_field]]))?,\s*(?:Group:\s*(?<Group_Name>[[sep_file_field]]))?,\s*(?:Server:\s*(?<Server_Name>[[sep_file_field]]))?,\s*(?<user>[[sep_file_field]]),\s*(?:Source\scomputer:\s*(?<Source_Computer_Name>[[sep_file_field]]))?,\s*(?:Source\sIP:\s*(?<Source_Computer_IP>[[sep_file_field]]))?,?\s*(?:Intensive\sProtection\sLevel:\s*(?<Intensive_Protection_Level>[[sep_file_field]]))?,?\s*(?:Certificate\sissuer:\s*(?<Certificate_Issuer>[[sep_file_field]]))?,?\s*(?:Certificate\ssigner:\s*(?<Certificate_Signer>[[sep_file_field]]))?,?\s*(?:Certificate\sthumbprint:\s*(?<Certificate_Thumbprint>[[sep_file_field]]))?,?\s*(?:Signing\stimestamp:\s*(?<Signing_Timestamp>[[sep_file_field]]))?,?\s*(?:Certificate\sserial\snumber:\s*(?<Certificate_Serial_Number>[[sep_file_field]]))? Hope this works for you. ... View more

Let see... if you have a API call like input like this, they work as a scripted input, and if they do not store a checkpoint value of some kind, it will end having duplicated data in the indexers because, for example, if you put the input on the indexers (assuming you have many of them, and you deploy from Master), or if you put the input on the heavy forwarders (assuming you have many of them deploying from Deployment Server and you want HA, you deploy on all them) .... you will get duplicated data. And so... I was thinking more on like a good implementation manual for productive environments on clustered splunk installations running on big companies. But then I saw the "Beta Test" in the .conf files ... I assume this is provided on a "best effort" policy, but any way I was asking more to Microsoft itself, to see if they have tried this on a clustered environment, and how have they managed this kind of issues. Since they use a lot of python scripts and such, they may have already faced this problems, and have already a "how to" or "best practices" on this. Thanks anyway for the manual reference, I have seen this before many times, I was looking more of a detailed reply from Microsoft side on how thir app is supposed to work, not from Splunk side. ... View more