you're welcome, glad it helped! ... View more

You'll have to make sure your data is CIM compliant. The root search for the datamodel is: (`cim_Network_Traffic_indexes`) tag=network tag=communicate You can build a search to vet the data that the datamodel is processing using the root search and showing the fields in tabular format: (`cim_Network_Traffic_indexes`) tag=network tag=communicate | table _time action app bytes bytes_in bytes_out dest dest_ip src src_ip Note that there's a lot more CIM fields, but since you're asking about bytes I'll focus on that. I'm assuming you have data that is tagged correctly, and that you have null values for bytes, bytes_in and bytes_out - if that's the case you have to make sure those fields exist in the source data, and that those values are all number values and not text values. All of the bytes value fields in the datamodel are calculated: The bytes calculation is case(isnum(bytes),bytes,isnum(bytes_in) AND isnum(bytes_out),bytes_in+bytes_out,1=1,null()) The bytes_in calculation is case(isnum(bytes_in),bytes_in,isnum(bytes) AND isnum(bytes_out),bytes-bytes_out,1=1,null()) The bytes_out calculation is case(isnum(bytes_out),bytes_out,isnum(bytes) AND isnum(bytes_in),bytes-bytes_in,1=1,null()) If your source data does not have the proper combination of bytes, bytes_in, and/or bytes_out or those values are not numeric values then you will wind up with null values for the bytes field values in your datamodel. If you have values that are non-numeric then you'll have to do some normalization work on your source data to convert them to numeric fields. ... View more

If two of the fields are working then my first guess would be that it's an issue with whitespacing in the regular expression. Try running a search that will give you the raw json results and add the following: | rex field=_raw "caller\"\s*:\s*\{\s*\"id\":\s*\"(?<test_callerid>[^\"]+)?\"\,\s*\"version\"\s*:\s*\"(?<test_callerversion>[^\"]+)?\"" See if you get the test_* fields extracted from the json. If the only thing at this point that needs to be tweaked is the regular expression, you might want to try tweaking it on regex101: https://regex101.com/r/4lGvKg/1 ... View more

If collecting local/RDP logon information will do the job for you, you can capture the Windows logs "Microsoft-Windows-TerminalServices-LocalSessionManager/Operational". If you're trying to track network logon sessions, I don't envy you. I worked on that for a bit and came to the conclusion we would never get any kind of useful aggregate reporting out of network session information from Windows logs. ... View more

Use btool on a system where the configuration has been deployed, not on the deployment server. I would probably test the field extraction part on a search head first, then when the extraction syntax has been deployed I would remove it from the testing search head and deploy it to the indexers or heavy forwarders and work on the indexed extraction. ... View more

If you run btool from the command line do you see the configuration items listed? /opt/splunk/bin/splunk cmd btool props list evento_srctyp --debug | less Replace /opt/splunk with whatever your $SPLUNK_HOME path is. I don't know if "evento_srctyp" is the actual sourcetype or if you masked it, but "splunk cmd btool props list {SOURCETYPE} --debug" will dump out the configuration. I wouldn't worry about indexing the field until you have the extractions working. When you can run the search: sourcetype=evento_srctyp | table _time host source callerid callerversion calledid calledversion and get results, then review link text and then work on the indexed extraction. ... View more

Is your ultimate goal to identify when users log on and log off? Are you trying to track interactive logons (local, RDP) only? I ask because if that's what you're trying to do I think there's a better way. Trying to build an idea of logon/logoff by object access is difficult. ... View more

If you're looking to do this with in-line searching, you'll want to use the rex command to extract text with regex. If you're looking to do field extraction, you'll need to read up on props.conf and transforms.conf Here is an in-line extraction example, using your message as a test value: | makeresults | eval message="<*Message>This is what I want to capture.<*/Message>" | rex field=message ">(?<message_value>[^\<]+)<" Regex101 is a great site for testing regular expression: https://regex101.com/r/ihEPfj/1 Regex is far too deep a subject to really cover in a Splunk answers post, but there's a number of good sites that will provide some basic information, including a Splunk page ( Splunk and Regular Expressions ) but when I really wanted to dig in to regex I picked up "Mastering Regular Expressions" by Jeffrey Friedl and it definitely helped. ... View more

*Update: Based on the data you provided in another comment I tweaked the regex. I would avoid lookaheads and lookbehinds if possible, especially with a big payload. It's too easy to have an poorly performing or broken regex. You also don't need to use the FORMAT command in transforms.conf if your regex is formatted to include the field names with the extractions. You can extract the caller and called ID and version fields with two stanzas, one for caller and one for called. [callerid] REGEX = caller\"\s*:\s*\{\s*\"id\":\s*\"(?<callerid>[^\"]+)?\"\,\s*\"version\":\s*\"(?<callerversion>[^\"]+)?\" [calledid] REGEX = called\":\s*\{\s*\"id\":\s*\"(?<calledid>[^\"]+)?\"\,\s*\"version\":\s*\"(?<calledversion>[^\"]+)?\" This was the inline search I used to test it: | makeresults | eval test="{\"info\": {\"eventSource\": \"RPA\", \"sourceType\": \"I\", \"status\": {\"code\": \"0000\", \"msg\": \"Inizio Schedulazione\", \"msgError\": \"\"}, \"transactionId\": \"66083\", \"traceId\": \"124021\", \"timestampStart\": \"2019-10-16T11:34:00.000Z\", \"timestampEnd\": \"null\", \"companyIDCode\": \"01\", \"channelIDCode\": \"\", \"branchCode\": \"\", \"searchFields\": [{\"VDI\": \"WPVRTM2004\"}, {\"PROCESSO\": \"Assegni\"}], \"annotation\": [{\"TIPO\": \"SCHEDULAZIONE\"}, {\"RISORSE POOL\": \"SI\"}], \"caller\": {\"id\": \"VWFM\", \"version\": \"1\", \"acronym\": \"WRPA0\"}, \"called\": {\"id\": \"Assegni\", \"version\": \"1\", \"acronym\": \"WRPA0\"}}, \"payLoad\": {\"output\": {\"encoding\": \"\", \"ccsid\": \"\", \"data\": \"\"}, \"input\": {\"encoding\": \"\", \"ccsid\": \"\", \"data\": \"\"}}}" | rex field=test "called\":\s*\{\s*\"id\":\s*\"(?<calledid>[^\"]+)?\"\,\s*\"version\":\s*\"(?<calledversion>[^\"]+)?\"" | rex field=test "caller\"\s*:\s*\{\s*\"id\":\s*\"(?<callerid>[^\"]+)?\"\,\s*\"version\":\s*\"(?<callerversion>[^\"]+)?\"" ... View more

Escaping backlashes with rex is strange. You could use three slashes: | rex field=_raw "Version=\\\x22(?<Version>.*?)\\\x" or use \x5c: | rex field=_raw "Version=\x5cx22(?<Version>.*?)\x5cx" ... View more

There could be a number of reasons - the first thing I would check is the permissions. Are you running Splunk as a service on the box? If so, are you running it as root or a user account? If you're running it as a service account you have to make sure the account as access to read the file. Are you ingesting other logs from this system, and if so are those being forwarded? I would normally start looking at the splunkd.log file on the host for an idea. grep "/var/log/snort/" /opt/splunkforwarder/var/log/splunk/splunkd.log assuming Splunk is installed in /opt/splunkforwarder, adjust the path as necessary. If you are running with non-root service account and want to verify permissions, then run this with an account that has sudo access: sudo su - splunk -s /bin/sh -c 'tail -n 1 $(find /var/log/snort/ -maxdepth 1 -type f -iname 'snort.log.*' -mtime -1 | tail -n 1)' This assumes that you're running Splunk using the service account "splunk", if you're using a different non-root service account then change "sudo su - splunk ..." to "sudo su - {your service account}" I also don't think you need the stanza "[monitor:///var/log/snort]" since you have "[monitor:///var/log/snort/snort.log.*]" good luck! ... View more

We've solved this by creating a group on the systems that has read access only to the log files we want to process. We configured the syslog service (rsyslog) to write logs with the file permissions 640 with the file group ownership set to the log review group, and we've added the Splunk forwarder service account to this group. You also have to mind your directory permissions to make sure that the Splunk service account can access the directory tree & enumerate/read files. Also make sure the forwarder web interface is disabled and the service account is properly restricted. ... View more

I'm not entirely clear how your messages are actually formatted - is "WindowsIdentity" the name of the field and also part of the field? are all these lines part of the same message and you want to skip the first value - "WindowsIdentity: IIS APPPOOL\login20.monster.com"? Maybe this will work: { ... base search ... } | rex max_match=100 field=WindowsIdentity "IIS APPPOOL\\\+\s?(?<App>.*?)\.monster\.com" | eval app_count=mvcount(App) | eval App=mvindex(App,1,app_count) ... View more

Have you tried btool - if you're running on Linux I would try something like $SPLUNK_HOME/bin/splunk btool props list --debug | grep "userEmail" Field aliasing, calculated fields, and lookups are all performed at search time after KV extractions, maybe there's an errant config that's overwriting your field. ... View more

I'm not sure if you should use a more complex single rex command, but you can: | rex "[Pp]roject (?:name: )?(?<ProjectName>[^\,\r\n]+)(?:\, Machine name\:(?<MachineName>[^\,]+)\, Status\:(?<JobStatus>[^\,]+)\, Backlog bytes\:(?<BacklogBytes>[0-9]+)\, Last consistency\:(?<LastConsistency>.*))?" | table ProjectName, MachineName, JobStatus, BacklogBytes, LastConsistency ... View more