About jcgever

jcgever · ‎11-21-2023

Got a customer wanting the UFs to send data to the forcepoint DLP and then to the intermediate heavy forwarder. The reason is to have the dlp mask PII even though we could have splunk do that. Any reason that that this data flow wouldn't work? An admin that I have to go through for the UF installs and the DLP setup tells me that forcepoint can't read splunk data sent to it which I don't believe. Apologies for possibly an easy question but any documentation to support the above architecture would be great.

jcgever · ‎03-15-2023

We did a linux patching cycle about a month ago. We have a 10 indexer 2 site cluster with 3:3 search and replication. I put the cluster into maintenance mode, stop splunk on an indexer, patch, reboot, wait until the indexer is up on the cluster manager and then repeat the cycle for the remainder of the indexers. Usually after the patching is done the bucket fixup tasks are a small amount and rapidly resolve. This past patching cycle we had a over 10k that are slowly resolving (maybe 30 a day). If I resync a bucket that number immediately drops by the amount I resync. I can only do 20 at a time because the cluster manager only allows 20 per page. That approach is silly with having to do that 10k times (currently sitting at 7k). I saw a community post about doing a rolling restart of the cluster will resolve this issue but it didn't. I did notice that there is 18 indexes (out of 126) that have access buckets. Wasn't sure if that affects anything. Is there a way to resync buckets more easily? Maybe 100 at a time without having to click through prompts?

jcgever · ‎09-16-2022

Awesome that did it @scelikok ! Thank you so much for your help.

jcgever · ‎09-15-2022

Been trying to work out the regex for the props and transforms for onbase logging and I keep hitting a brick wall. Here is a sample log of what I have been working on: <?xml version="1.0" encoding="utf-8"?> <diagnosticsLog type="error-profile" startDate="07/28/2022 01:10:20">  <columns> <column friendlyName="time" name="time" /> <column friendlyName="Result" name="Result" /> <column friendlyName="Module" name="Module" /> <column friendlyName="Class" name="Class" /> <column friendlyName="SourceFile" name="SourceFile" /> <column friendlyName="Method" name="Method" /> <column friendlyName="SourceLine" name="SourceLine" /> <column friendlyName="Severity" name="Severity" /> <column friendlyName="MachineName" name="MachineName" /> <column friendlyName="IpAddress" name="IpAddress" /> <column friendlyName="ErrorId" name="ErrorId" /> <column friendlyName="ProcessID" name="ProcessID" /> <column friendlyName="ThreadID" name="ThreadID" /> <column friendlyName="TimeSpan" name="TimeSpan" /> <column friendlyName="User" name="User" /> <column friendlyName="HTTPSessionID" name="HTTPSessionID" /> <column friendlyName="HTTPForward" name="HTTPForward" /> <column friendlyName="SessionID" name="SessionID" /> <column friendlyName="SessionGUID" name="SessionGUID" /> <column friendlyName="Datasource" name="Datasource" /> <column friendlyName="Sequence" name="Sequence" /> <column friendlyName="LocalSequence" name="LocalSequence" /> <column friendlyName="Message" name="Message" /> <column friendlyName="AppPoolName" name="AppPoolName" /> </columns> <rows> <row> <col name="time">07/28/2022 01:10:20</col> <col name="TimeSpan">N/A</col> <col name="ThreadID">0x0000000A</col> <col name="User"></col> <col name="HTTPSessionID"></col> <col name="HTTPForward"></col> <col name="SessionGUID"></col> <col name="SessionID">0</col> <col name="Datasource">OnBaseQA</col> <col name="AppPoolName"></col> <col name="IpAddress"></col> <col name="MachineName"></col> <col name="Result">0x00000000</col> <col name="Message">FileLoadException: File [D:\Program Files\Hyland\Services\Distribution\System.Runtime.CompilerServices.Unsafe.dll] Message [Could not load file or assembly 'System.Runtime.CompilerServices.Unsafe, Version=4.0.3.0, Culture=neutral, PublicKeyToken=abcd1234' or one of its dependencies. The located assembly's manifest definition does not match the assembly reference. (Exception from HRESULT: 0x901234)]</col> <col name="Module">Hyland.Core</col> <col name="Class">AssemblyRegistration</col> <col name="Method">ReadAssemblyXml</col> <col name="SourceFile"></col> <col name="SourceLine">0</col> <col name="Severity">Error</col> <col name="ErrorId"></col> </row> I'm trying to cut out or blacklist the first portion with the column naming and only grabbing the col name in between the "" with the corresponding data. I'm very new to manipulating data like this and am just starting to understand regex. These are my attempts on props and transforms: Transforms: [log_time] DEST_KEY = MetaData:Sourcetype REGEX = <col name="time">(.*)<\/col> FORMAT = time::"$1" [time_span] DEST_KEY = MetaData:Sourcetype REGEX = <col name="TimeSpan">(.*)<\/col> FORMAT = time_span::"$1" [thread_id] DEST_KEY = MetaData:Sourcetype REGEX = <col name="ThreadID">(.*)<\/col> FORMAT = thread_id::"$1" [user] DEST_KEY = MetaData:Sourcetype REGEX = <col name="User">(.*)<\/col> FORMAT = user::"$1" [http_session_id] DEST_KEY = MetaData:Sourcetype REGEX = <col name="HTTPSessionID">(.*)<\/col> FORMAT = http_session_id::"$1" [http_forward] DEST_KEY = MetaData:Sourcetype REGEX = <col name="HTTPForward">(.*)<\/col> FORMAT = http_forward::"$1" [session_guid] DEST_KEY = MetaData:Sourcetype REGEX = <col name="SessionGUID">(.*)<\/col> FORMAT = session_guid::"$1" [session_id] DEST_KEY = MetaData:Sourcetype REGEX = <col name="SessionID">(.*)<\/col> FORMAT = session_id::"$1" [datasource] DEST_KEY = MetaData:Sourcetype REGEX = <col name="Datasource">(.*)<\/col> FORMAT = datasource::"$1" [app_pool_name] DEST_KEY = MetaData:Sourcetype REGEX = <col name="AppPoolName">(.*)<\/col> FORMAT = app_pool_name::"$1" [ip_address] DEST_KEY = MetaData:Sourcetype REGEX = <col name="IpAddress">(.*)<\/col> FORMAT = ip_address::"$1" [machine_name] DEST_KEY = MetaData:Sourcetype REGEX = <col name="MachineName">(.*)<\/col> FORMAT = machine_name::"$1" [result] DEST_KEY = MetaData:Sourcetype REGEX = <col name="Result">(.*)<\/col> FORMAT = result::"$1" [message] DEST_KEY = MetaData:Sourcetype REGEX = <col name="Message">(.*)<\/col> FORMAT = message::"$1" [module] DEST_KEY = MetaData:Sourcetype REGEX = <col name="Module">(.*)<\/col> FORMAT = module::"$1" [class] DEST_KEY = MetaData:Sourcetype REGEX = <col name="Class">(.*)<\/col> FORMAT = class::"$1" [method] DEST_KEY = MetaData:Sourcetype REGEX = <col name="Method">(.*)<\/col> FORMAT = method::"$1" [source_file] DEST_KEY = MetaData:Sourcetype REGEX = <col name="SourceFile">(.*)<\/col> FORMAT = source_file::"$1" [source_line] DEST_KEY = MetaData:Sourcetype REGEX = <col name="SourceLine">(.*)<\/col> FORMAT = source_line::"$1" [severity] DEST_KEY = MetaData:Sourcetype REGEX = <col name="Severity">(.*)<\/col> FORMAT = severity::"$1" [error_id] DEST_KEY = MetaData:Sourcetype REGEX = <col name="ErrorId">(.*)<\/col> FORMAT = error_id::"$1" Props: [error_profile] SHOULD_LINEMERGE=true LINE_BREAKER=([\r\n]+) NO_BINARY_CHECK=true TRANSFORMS-set = log_time, time_span, thread_id, user, http_session_id, http_forward, session_guid, session_id, datasource, app_pool_name, ip_address, machine_name, result, message, module, class, method, source_file, source_line, severity, error_id

jcgever · ‎01-29-2021

We are moving to a new storage array for our frozen data. Our old array is setup where our indexers have separate nfs mounts for the frozen data. Would there be an issue of having 5 indexers pointing to separate folders on one NFS mount or would it be best for each indexer to have separate NFS mounts for their frozen storage?

jcgever · ‎10-26-2020

The search head that our security team uses is filling up the /opt/splunk/var/lib/splunk/kvstore/. The directory is about 400+GB. To be honest I'm not sure what role the kvstore plays but I'm feeling like it shouldn't be taking up this much space. Any tips on how to clean this up without messing with the functionality?

jcgever · ‎02-13-2020

We have a few users that need access to application logs. We have our active directory admins create a group and once they create that group it shows up in splunk for us to add a role to. The latest group to be created shows up in the "Access controls » Authentication method » LDAP strategies » LDAP Groups" page but once I try to add a role other than "user" it doesn't show as added in the UI even when the message at the top of the screen says the role has been added. The users can't search any logs that they should have access through the new role created for the new LDAP Group. What's odd is that the /opt/splunk/etc/system/local/authentication.conf has the new role added to the new LDAP Group. looking in splunkd.log there is this message: 02-06-2020 10:58:07.296 -0500 WARN UserManagerPro - Strategy="Splunk": the group="SPL_DIGITAL" was not found on the LDAP server. Suggest to remove it from the role map to save server loading time. Not sure what to do. Not sure if this is a problem with AD or with splunk.

jcgever · ‎10-25-2019

We are in the middle of trying to split the stream of data between splunk and a third party. In order to do this we are going to have to push out a custom universal forwarder to about 1000 servers and black list them in a server group on the deployment server. Is there a limit to the amount of server names that we can blacklist?

jcgever · ‎07-26-2019

I have a search that tells me if an index hasn't received data from a log on a server. This allows me to monitor the splunk environment and make sure that everything it working as expected. The only problem with the search is when it pertains to application logs that aren't written to everyday. I get a lot of false positives. I have also created an app that does an ls -al on all the inputs.conf in etc/apps/ and etc/system/local and sends the results to an index (index_check_unix). Combining the information of these 2 items is where I'm having the trouble. I either want to combine the search and the additional index and only get the servers that are having issues or just having a separate column that has the output of the index_check_unix based on the servers that show up in the report. Below is my search for unix servers I have a separate search for windows: | tstats latest(_time) AS lastTime WHERE index=* BY host index sourcetype source| eval current=now() | eval age_min=round((current-lastTime)/60,2) | rangemap field=age_min low=0-720 elevated=721-1440 | search range!=low | search range!=elevated | stats max(current) AS "Current Time" values(index) AS index values(sourcetype) AS sourcetype values(source) AS Location list(lastTime) AS "Latest Event" list(range) AS Status by host | convert ctime(*Time) ctime("Latest Event")

jcgever · ‎03-22-2019

Thanks! I didn't think to use tstats. Works like a charm now.

jcgever · ‎03-21-2019

I'm new to xml and building dashboards in splunk. What I'm trying to do is build a dashboard that takes information from the user through drop down menus that build on each other. I've gotten it almost figured out but the last drop down menu doesn't populate with any results. Any pointers? here is what I have so far: <form> <label>Environment search</label> <fieldset submitButton="false"> <input type="time" token="datetime" searchWhenChanged="true"> <label>Date and Time</label> <default> <earliest>-15m</earliest> <latest>now</latest> </default> </input> <input type="dropdown" token="env" searchWhenChanged="true"> <label>Environment</label> <choice value="*">all</choice> <choice value="*l1q*">qa</choice> <choice value="*l1d*">dev</choice> <choice value="*l1s*">stg</choice> <choice value="*1p*">prod</choice> <fieldForLabel>env</fieldForLabel> <fieldForValue>env</fieldForValue> <search> <query/> <earliest>$datetime.earliest$</earliest> <latest>$datetime.latest$</latest> </search> </input> <input type="dropdown" token="host"> <label>Host</label> <fieldForLabel>host</fieldForLabel> <fieldForValue>host</fieldForValue> <search> <query>host=*$env$ | dedup host</query> <earliest>$datetime.earliest$</earliest> <latest>$datetime.latest$</latest> </search> <choice value="*">all</choice> <default>*</default> <initialValue>*</initialValue> </input> <input type="dropdown" token="apps" searchWhenChanged="true"> <label>Apps</label> <fieldForLabel>apps</fieldForLabel> <fieldForValue>apps</fieldForValue> <search> <query>host=$host$ index=* |dedup index host</query> <earliest>$datetime.earliest$</earliest> <latest>$datetime.latest$</latest> </search> <choice value="*">all</choice> <default>*</default> <initialValue>*</initialValue> </input> </fieldset> <row> <panel> <event> <title>logs</title> <search> <query>host=$host$ index=$apps$</query> <earliest>$datetime.earliest$</earliest> <latest>$datetime.latest$</latest> </search> <option name="list.drilldown">none</option> <option name="refresh.display">progressbar</option> </event> </panel> </row> </form>

Posts	11
Solutions	0
Karma Given	3
Karma Received	0
Member Since	‎09-15-2016

Online Status	Offline
Date Last Visited	‎11-21-2023 08:52 AM

Sending splunk data through a dlp to a heavy forwa...

Why do Fixup tasks still present after rolling res...

Help in parsing logs- How to cut out or blacklist ...

separate nfs mounts for frozen vs. one NFS mount f...

Kvstore is filling up on search head

Newly created LDAP group not accepting created rol...

Is there a limit to the amount of servers a deploy...

column in search needs to be populated by another ...

using multiple tokens to populate a dashboard