Are you simply trying to sum a total count from all the distinct title counts? Ie, adding together all counts in the column? You could try using addcoltotals: https://docs.splunk.com/Documentation/Splunk/6.5.2/SearchReference/Addcoltotals If that doesn't answer your question, could you describe a little further what you're looking for? ... View more

Does this answer address your question? https://answers.splunk.com/answers/279/does-splunk-index-gzip-files.html ... View more

Is it possible to provide the xml config? ... View more

I'm not sure if I understand completely what your request is. Are you saying Splunk is extracting different product names from your events as their own individual fields? Could you provide a few samples of your events, and a screen shot? I may be able to help, but I'd need to see the data first. ... View more

A couple of things. This article has a lot of good stuff for improving searches: http://docs.splunk.com/Documentation/Splunk/latest/Search/Writebettersearches Aside from that article, you could try some of these things: After the base search, try piping to the fields command to specify only the fields you're using. For example, if your search has something like stats count by fieldA, fieldB, FieldC or a table fieldA, fieldB, FieldC, try something like this: ... | fields fieldA, fieldB, FieldC | rest of your search This prevents unnecessary field extractions and improves performance. Try to only specify fields you're using. Can you narrow the search terms down any further? Splunk likes when you're specific. Is there a specific sourcetype associated with the index? Is there another field consistent throughout all the results? How big of a time frame are you running this search over? You could consider running the search over smaller periods of time, and either write it to a report or write the output to a namespace. Then call it using tstats. You could try scheduling a report and accelerating it. ... View more

Since your original query simply has a chart count at the end, and you want it to be a timechart, why not change the end terms to timechart count? You can change the span of 7 days (7d) into something bigger or smaller depending on how you want the view to look. Given that you're going 90 days back, I figured a week span was appropriate. source=duo extracted_eventtype=authentication result="SUCCESS" earliest=-90d | eval Period=if(_time>=relative_time(now(),"-7d"),1,2) | stats max(Period) as periods by username | where periods=1 | timechart span=7d count ... View more

Thanks for sharing! Learned something new. ... View more

Yikes! 0 for 2. Alright, lets try this. Is there any way you can provide me with a sample of the data? Or is this secret squirrel stuff? What filetype(s) reside inside? Is the data inside the files always formatted the same way or does it vary? Are you creating these gzipped files manually or are they being exported in an automated fashion from another source? Do these logs reside on a Linux box? Surprisingly, I've had to do something like this (kind of ) for zip files exported from an antivirus. I wrote a PowerShell script to unzip the files, convert them to txt, append a timestamp, and feed it to Splunk. In your case, it may be a little more work, but I think it can be done. Need more infos though. ... View more

I think I see what you’re asking now. Apologies if I misunderstand again. Just to clarify, you want to use the date_modified in the event as the timestamp of the event (_time)? I didn’t realize this at first, but there is no year in the timestamp of your events, so Splunk is attempting to extract something that isn’t there. You'd have to specify how far into the event you want Splunk to look in order for Splunk to use the current year appended to the month and day. I took your events and ingested them in my own Splunk instance to test this. I had to keep changing the times in the logs so I didn't have to clear the fish bucket, so the dates and times are different, but the format is the same. It took me a few tries but I ended up getting it working. I used the following stanza in my props.conf [testtime] TIME_PREFIX = datasvcs\s+\d+\s+ TIME_FORMAT = %b %d %H:%M MAX_TIMESTAMP_LOOKAHEAD = 65 Raw Logs with timestamps: After tabling the _time with the raw events: Hope that helps! ... View more

Did you have issues receiving these logs before? Or is this the first time? Did you make any changes to configuration files or upgrade anything in Splunk? Since I do not know your environment, I cannot verify with what you’ve given me whether this is an issue with Splunk, Cisco, or an intermediary network/security product in between. Going through this systematically will allow you to determine where the issue is. Try going through some troubleshooting steps. Start tracing it from the source to the destination. Try sending an email from your personal email to your work email and use that to trace the event from start to finish. Verify whether the event appears in the logs from the appliance. If yes, Verify if the appliance is forwarding those logs to the syslog server. If yes, Find the event in the syslog server and verify if the event was sorted (based on your syslog configuration file rules) into the appropriate directory. If yes, Verify the Splunk forwarder on the syslog server is configured to monitor that directory. If yes, Determine if the forwarder sent that event to the indexer. At this point you should be able to simply search for the event. Search all indexes, index=* youremail@yourdomain.com , for example. Hope that helps. ... View more

1.) Have you verified whether or not inbound email is a configured log subscription in the ESA? 2.) Have you verified whether or not the indexers are ingesting inbound email logs from the ESA? 3.) Have you verified the logs are not going to a different sourcetype or generic sourcetype? ... View more

Have you tried using TIME_PREFIX? http://docs.splunk.com/Documentation/Splunk/6.5.1/Data/Configuretimestamprecognition Try something like TIME_PREFIX = datasvcs\s+\d+\s ... View more

You're very welcome! Hmm, I don't quite understand what you mean by not having the field. The recipient_count field is not a field from the exchange logs. It is a field created by you and I at search time by renaming the field after the transforming command "stats", for example: dc(RecipientAddress) AS recipient_count by SenderAddress. Whenever you place a field inside of a function, like dc(), or count() from your example, you can choose whether or not to rename the field using the "as" clause. I'm not sure why the stats eval condition is needed in this context? Is this a search you're trying to combine or change from a previous admin? It may help to provide us with a sanitized sample of the data and explain what is you're trying to do. In the second search you mentioned expanding the search returns no results. The search is not working because it isn't composed correctly. The first part of the search is creating a statistical summary (stats count) following dedup (deleting duplicates) of the MessageId field. The arguments for the stats command is returning the count of MessageId as the field recipient_count. This search by itself will only return a single field: recipient_count, which for me, appears to return the resulting count of events with MessageId in them, renamed as the field "recipient_count". From here, you are trying to create an ordered table of the previous statistical summary value (the value of recipient_count) with other fields not included in the previous stats command, which is not possible to do. The timechart will also not work should you remove the table because timecharts are created with the timestamps of the events. ... View more

Does Splunk have permission to run the script? Try looking for issues with permissions in the internal log: index=_internal yourscriptname ... View more

In your search, you’re attempting to table the _time field when _time is not included as a field after your stats command, and also trying to create a timechart without the _time field as an included field after your stats command. You will not be able to retain that table after piping to a timechart. Not sure if you wanted a table or a timechart? Since your final pipe is to a timechart, I went with that first. If you want the total emails sent per user per day with a timechart, try this: index=exchange NOT Status=Quarantined NOT Status=Failed SenderAddress=*.xyz.com | timechart span=1d dc(RecipientAddress) AS recipient_count by SenderAddress This should provide you with a timechart of the distinct count of recipients by the sender with a span of 1 day over whatever time period you choose. I noticed you are searching on exchange, and also specifying a wildcard on the SenderAddress field with the domain, which I interpreted as you wanting to look at a sender(s) email activity from a domain. With an index like exchange, depending on the size of your organization, this timechart might be difficult to read if you don’t specify a sender or a domain. The results in the timechart will truncate if you have more than 1000 senders in the timeframe. Now, if you didn’t want a timechart and instead want a table with the information from the sender, try something like this: index=exchange NOT Status=Quarantined NOT Status=Failed SenderAddress=*.xyz.com | stats count(RecipientAddress) AS recipient_total, values(Subject) AS Subject, values(MessageId) AS MessageId, values(Size) AS Size by SenderAddress | table SenderAddress, recipient_total, Subject, MessageId, Size Counting the values of time for the search above can get messy since it will just group all the times together, but you could try using a function like first(_time) or last(_time) in front of the values if you want to know when the oldest or newest event occurred. For example, index=exchange NOT Status=Quarantined NOT Status=Failed SenderAddress=*.xyz.com | stats count(RecipientAddress) AS recipient_total, first(_time) AS _time, values(Subject) AS Subject, values(MessageId) AS MessageId, values(Size) AS Size by SenderAddress | table _time, SenderAddress, recipient_total, Subject, MessageId, Size This would give you the most recent event’s time out of all the events. ... View more

You can also install and configure sysmon. https://technet.microsoft.com/en-us/sysinternals/sysmon http://blogs.splunk.com/2014/11/24/monitoring-network-traffic-with-sysmon-and-splunk/ The event code that would interest you is EventCode=3 You're also able to see which application is making the DNS query and any command line entries initiating the communication. I'm using it on my home lab and have worked contracts in the past where customers were leveraging sysmon logs with Splunk. If you choose to use this option, make sure you filter events properly, both in the sysmon.xml config and in your inputs.conf (for Windows events) and/or prop.conf/transforms.conf for sending noisy events to a nullqueue. Ensure you test it first. Sysmon can generate an absurd amount of logs if not configured correctly. ... View more

Glad I could help 🙂 ... View more

Could you post a sample of the event data? ... View more

A few things: I think your script attribute value in the stanza might be incorrect. The forward slashes should be backslashes in a windows environment: http://docs.splunk.com/Documentation/AddOns/released/MSPowerShell/Configuration Try the full path name, no dots. It also appears you are missing an opening bracket [ in front of the PowerShell in the stanza header. If it still isn't working after that: Have you checked splunkd.log for that stanza/script for any errors? If no, try that. You can do this via the search head by looking in index=_internal sourcetype=splunkd Do these two different scripts provide two unique outputs? Ie, do not have the same hash value? ... View more

Sure, try something like this: @echo off powershell.exe -ExecutionPolicy bypass -file "X:\Path\to\your\script.ps1" ... View more

Yeah, using the same method you have above. Just replace the .ps1 with the .bat which calls the .ps1. You can keep the same sourcetypes or change them at your leisure. ... View more

This happened to me a few months ago. I fixed it by creating a batch script to call the powershell script. ... View more