Thanks for the Slack link. Dashboard Studio has gotten better at time zone detection relative to SplunkWeb user preferences and browser locale, but it would be clearer if it worked the same way every time, i.e., it used a zone offset if provided or else used the SplunkWeb offset. There are other ways to confuse Dashboard Studio, e.g.: | makeresults | fieldformat _time=strftime(_time, "%d-%b-%Y") _time is detected as a "time" field, but Dashboard Studio displays "Invalid Date" because the underlying JavaScript object is not a Date object. ... View more

Hi @yuanliu, Did you come up with a satisfying answer? From my brief experiments, Dashboard Studio appears to cast or convert the field value to a JavaScript Date object using the Date.parse() method. If the string is in a date-time format recognized by the browser's JavaScript engine*, a valid Date object is returned, and Dashboard Studio gives the label or value a type of "time"; otherwise,  the label or value may have a type of "number," "string," or "unknown" depending on additional tests. The behavior changes when the search coerces a variant field to a specific type with, e.g., the replace() or tostring() functions. If I'm poking around in the right spot, the logic is: IF Finite AND Number THEN // x.isFinite() && x.isNumber(x)     "number" ELSE IF Date THEN // !Number.isNan(Date.parse(x))     "time" ELSE IF String THEN // Number.isNaN(Date.parse(x)) && typeof x == "string"     "string" ELSE     "unknown" END IF There is necessarily some conversion from Splunk field to JavaScript variable when Dashboard Studio parses search results in client-side JavaScript. The logic contradicts the behavior change, though, as date-time values coerced to strings should pass the (inferred) !Number.isNan(Date.parse(x)) test and have a type of "time" and not "string." * Different browsers support different date-time string formats, but the standard formats should be universal. ... View more

I love engineering/systems-oriented dashboards and applied puzzle solving, but when I try to do something "pretty" ... the Titanic flute fail meme comes to mind. My favorite dashboards tend to be simple things with 3-5 metrics (principal component analysis maybe?) that say this is your root cause.  ... View more

I'm also a bit like everyone else: I prefer Simple XML; however, that's mostly because it's what I've used daily for ... too many years. Dashboard Studio iterates significantly with every release. Splunk Enterprise 10.4 and Splunk Cloud Platform 10.4.2604 added support for custom visualizations, for example. The product manager for Dashboard Studio and other SplunkWeb features posts blog updates and (presumably) monitors Answers. I know they welcome friendly, constructive feedback. That said, Splunk Enterprise 9.4 is nearly three years old now and more than a little out of date with respect to Dashboard Studio. ... View more

Hi @eholz1, I'll add one more example that works in Splunk Enterprise 9.4.12 and sets token values directly from a column chart. The token value you're looking for is probably row.<fieldname>.value, where <fieldname> corresponds to the over/row-split field in the chart command or the x-value in the xyseries command. E.g.: index=_internal source=*splunkd.log* | chart limit=10 usenull=f useother=f count over log_level by component index=_internal source=*splunkd.log* | stats count by log_level component | xyseries log_level component count => row.log_level.value You copy and paste the source below or use it as a reference to define the dashboard using Dashboard Studio. There's no source-only magic in the dashboard. { "title": "ds_drilldown", "description": "", "inputs": { "input_global_trp": { "options": { "defaultValue": "-24h@h,now", "token": "global_time" }, "title": "Global Time Range", "type": "input.timerange" } }, "defaults": { "dataSources": { "ds.search": { "options": { "queryParameters": { "earliest": "$global_time.earliest$", "latest": "$global_time.latest$" } } } } }, "visualizations": { "viz_TbwbICq3": { "dataSources": { "primary": "ds_zDetExHq" }, "options": {}, "type": "splunk.table" }, "viz_ZhY8abJJ": { "options": { "markdown": "**chart_name:** $chart_name$\n\n**chart_value:** $chart_value$\n\n**chart_row_log_level_value:** $chart_row_log_level_value$" }, "type": "splunk.markdown" }, "viz_eASiDDzL": { "dataSources": { "primary": "ds_AIqBMriG" }, "eventHandlers": [ { "options": { "tokens": [ { "key": "name", "token": "chart_name" }, { "key": "value", "token": "chart_value" }, { "key": "row.log_level.value", "token": "chart_row_log_level_value" } ] }, "type": "drilldown.setToken" } ], "options": {}, "type": "splunk.column" } }, "dataSources": { "ds_AIqBMriG": { "name": "Search_1", "options": { "query": "index=_internal source=*splunkd.log*\n| chart limit=10 usenull=f useother=f count over log_level by component", "queryParameters": {} }, "type": "ds.search" }, "ds_zDetExHq": { "name": "Search_2", "options": { "query": "index=_internal source=*splunkd.log* log_level=$chart_row_log_level_value|s$\n| table _time log_level component _raw" }, "type": "ds.search" } }, "layout": { "globalInputs": [ "input_global_trp" ], "layoutDefinitions": { "layout_1": { "options": { "height": 960, "width": 1440 }, "structure": [ { "item": "viz_eASiDDzL", "position": { "h": 400, "w": 1440, "x": 0, "y": 0 }, "type": "block" }, { "item": "viz_ZhY8abJJ", "position": { "h": 87, "w": 1440, "x": 0, "y": 400 }, "type": "block" }, { "item": "viz_TbwbICq3", "position": { "h": 400, "w": 1440, "x": 0, "y": 487 }, "type": "block" } ], "type": "grid" } }, "options": {}, "tabs": { "items": [ { "label": "New tab", "layoutId": "layout_1" } ] } } }   ... View more

Hi @user5, The order of fields in current NetScaler output doesn't match what the add-on expects. I don't have a sample I can share, but if you can post an obfuscated sample of a raw event, e.g., SSLVPN LOGIN, from your environment, we can help with field extraction fixes. ... View more

Dashboard Studio in Splunk Enterprise 10.2, Splunk Cloud Platform 10.1.2507, and later supports JSONata expressions. I drafted an example here, but I haven't seen them used in the wild. ... View more

Hi @las, The article you referenced is for Splunk SOAR, but if you want to explore the PostgreSQL environment shipped with Splunk Enterprise, you can install PostgreSQL 17 tools on your Linux server. Make a backup of your Splunk Enterprise installation before proceeding. I recommend using a restored backup running in a sandbox. I wouldn't do any of this on a production deployment server. This process is for Red Hat Enterprise Linux 10. It's similar for other distributions. PostgreSQL publishes a handy guide at https://www.postgresql.org/download/linux/. 1. Disable the RHEL postgresql module: sudo dnf module disable postgresql 2. Install the PostgreSQL repository: sudo dnf install https://download.postgresql.org/pub/repos/yum/reporpms/EL-10-x86_64/pgdg-redhat-repo-latest.noarch.rpm 3. Install PostgreSQL 17 tools: sudo dnf install postgresql17 4. Connect to PostgreSQL: psql "host=localhost port=5432 dbname=postgres user=postgres_admin" Your local postgres_admin password is encrypted in $SPLUNK_HOME/etc/system/local/passwords.conf. You can decrypt the value with: $SPLUNK_HOME/bin/splunk show-decrypted --value '<password>' where <password> is the encrypted value in passwords.conf. Or all together: psql "host=localhost port=5432 dbname=postgres user=postgres_admin password=$($SPLUNK_HOME/bin/splunk show-decrypted --value "$($SPLUNK_HOME/bin/splunk btool passwords list credential:postgres:postgres_admin: | grep '^password\s*=' | sed 's/^password[[:space:]]*=[[:space:]]*//')")" From here, you can list databases with \l, list users with \du, or run other commands as needed. Can you damage the installation beyond repair? YES. Verify your backup before making changes. It would be best to proceed with the help of Splunk support. If the issue you have is reproducible, they can fix it for other customers, too. ... View more

Hi @avadhi, Your question is a few months old, but did you find a solution? Splunk Add-on for Amazon Web Services (AWS) collects the following AWS/DynamoDB metrics by default after the namespace is added to a CloudWatch input: "AWS/DynamoDB": [ "ConditionalCheckFailedRequests", "ConsumedReadCapacityUnits", "ConsumedWriteCapacityUnits", "OnlineIndexConsumedWriteCapacity", "OnlineIndexPercentageProgress", "OnlineIndexThrottleEvents", "ProvisionedReadCapacityUnits", "ProvisionedWriteCapacityUnits", "ReadThrottleEvents", "ReturnedBytes", "ReturnedItemCount", "ReturnedRecordsCount", "SuccessfulRequestLatency", "SystemErrors", "ThrottledRequests", "UserErrors", "WriteThrottleEvents", ] If you don't see your metrics when editing the input configuration, try upgrading your Splunk Add-on for AWS installation. If you do see the metrics, verify you can see data in CloudWatch using the AWS Management Console or the AWS CLI. ... View more

Hi @ankit13, I recommend Splunk Universal Forwarder 9.3.x on Windows Server 2016. Splunk Universal Forwarder 9.3.13 was just released. Splunk has been inconsistent on whether more recent versions of the forwarder are fully tested and supported on Windows Server 2016. Both products will be end of life soon. As others have said, make sure your TLS configuration is consistent on both the client (deploymentclient.conf and server.conf) and server (server.conf). You can verify connectivity using the forwarder's bundled version of OpenSSL. From an elevated command prompt:  cd ; & "C:\Program Files\SplunkUniversalForwarder\bin\splunk.exe" cmd openssl s_client -connect <host>:8089 -CAfile "C:\Program Files\SplunkUniversalForwarder\etc\auth\cacert.pem" -tls1_2 -verify_return_error Replace <host> with your deployment server hostname, FQDN, or IP address. If you use an IP address, you can add the -servername parameter to openssl; set it to the common name or a valid subjectAltName value in the server certificate. If you need to use an IP address, check your name resolution configuration (DNS and/or %SystemRoot%\System32\drivers\etc\hosts). The "cd ; &" syntax just lets the command run as-is under both PowerShell and cmd.exe. A command that fully emulates your current Splunk configuration requires a copy of your forwarder's deploymentclient.conf and server.conf (remove any encrypted or plaintext secrets before posting). On failure, OpenSSL will print useful error messages and help isolate root cause. ... View more

I don't think you'll reach 100% without duplication. I would model the delay as a random variable and plan for missing samples. Dynatrace support may be able to provide more details about how their events are queued and delivered. ... View more

Hi @karl_lbg, Without modifying the modular input scripts, you can select a Dynatrace Collection Interval value larger than your input interval. Since the Dynatrace Collection Interval overlaps with the previous input interval, you may see duplicate events. It's a trade-off vs. missing events. You can optimize by using a Dynatrace Collection Interval value that approximates your Dynatrace environment's lag. For example, if your lag is 2 minutes and your input interval is 5 minutes (300 seconds), the closet Dynatrace Collection Interval is 10 minutes. Each interval would collect from -10 minutes to -0 minutes of data and pick up any events the prior interval missed. If you want to modify the modular input code, you can store the last observed timestamp as a checkpoint, use that value in the API startTimestamp parameter, and set the API endTimestamp parameter to the current time less some backoff. Time-based checkpoints are usually probabilistic, and there's always a chance you'll still miss events. The internal app developer (see Splunkbase) at Splunk may be open to suggestions. Have you contacted them directly? If you want to write your own input to retrieve logs and metrics without writing a lot of code, you may prefer the REST API Modular Input and the Dynatrace Grail service. ... View more

I have a bit of free time, so I checked: with DEBUG logging enabled, LineBreakingProcessor appears to be skipped when the _linebreaker key is present. With _linebreaker: S2SReceiver S2SReceiverEvents (_raw = line) UTF8Processor AggregatorMiningProcessor regexExtractionProcessor (_MetaData:Index transform) Without _linebreaker: S2SReceiver S2SReceiverEvents (_raw = chunk) UTF8Processor LineBreakingProcessor AggregatorMiningProcessor DateParser regexExtractionProcessor (_MetaData:Index transform) Using a route with has_key:_linebreaker:typingQueue jumps from S2SReceiverEvents to regexExtractionProcessor with respect to transforms. It's probable I've looked at all of this before when verifying earlier implementations. Instant recall would be nice. ... View more

Hi @Enzo54, You will find the same problem in 10.x. Don't be afraid to increase the size of max_memtable_bytes in this or other cases. The setting is there to help you get the best performance out of frequently used large lookup files. While null is a valid UTF-8 character, it poses a challenge for internal functions expecting null-terminated strings, whether they're in splunkd, Python's CSV reader, or elsewhere. I work occasionally with binary data in Splunk, so I wouldn't assume your null bytes are corruption as @kml_uvce has done, but their advice is otherwise sound: preprocess your lookup files and use a surrogate character to represent null. Splunk uses \x00 itself when sanitizing _raw, but you may prefer the URL-encoded %00, the UTF-8 encoding for ␀ (the byte sequence E29080), or some other value depending on your source representation. Whatever you choose, deserializing the data is simple with the rex command using mode=sed or the eval command using the replace() and urldecode() functions. ... View more

Logs? What is this? Splunk? I'd use a decompiler or disassembler. 😉 Or more realistically, I'd do black box testing across multiple (but serialized) tiers and analyze S2S traffic for changes in event boundaries. ... View more

I'm old, too. My kids remind me daily. Answers is already well served by conforming solutions and generative AI. Witness the ratio of trustee replies to questions and the downtrend in traffic. I spend my days mired in compliance-ridden, risk-averse systems. My answers here are both an escape and a way to contribute to a community of practitioners I've enjoyed engaging across venues since Splunk 4. ... View more

It would be nice to know without reverse engineering how parsingQueue and aggQueue behave internally when the _linebreaker key is already present. Are LINE_BREAKER and SHOULD_LINEMERGE ignored in the second parsing and aggregation passes, respectively? ... View more

I can't speak to customers not documenting their own stuff, but I do think people should use features, however obscure. One downside: public Splunk consulting documentation implies (and LLMs, if you use them, infer) that RULESET is only supported on heavy forwarders and indexers. It's not true, but if it sounds plausible, folks will accept it as fact and never question it. Edit: If anyone does try this in a supported environment and encounters problems, I would encourage them to contact Splunk support and get an answer on the record. I sound cavalier, but I'm no more or less conservative than my engagements want me to be. ... View more

Hi All, Resurfacing this old thread to post an alternative solution. This will increase CPU and memory utilization on the forwarder, but we can combine force_local_processing with RULESET to create a forwarder-side ingest action: # props.conf [source::WinEventLog:*] force_local_processing = true RULESET-winventlog_ignore_older_than = winventlog_ignore_older_than # transforms.conf [winventlog_ignore_older_than] # send events older than 30 days * 86,400 seconds/day = 2,592,000 seconds to nullQueue INGEST_EVAL = queue=if((time() - _time > 2592000), "nullQueue", queue) # easier to read but slower #INGEST_EVAL = queue=if((time() - _time > relative_time(time(), "-30d"), "nullQueue", queue) STOP_PROCESSING_IF = queue == "nullQueue" This should work on Splunk Universal Forwarder 9.0 or later, but I've only briefly tested 10.4. Edit: This may only work in 9.2 and later. I no longer use 9.0 or 9.2 and haven't tested them. ... View more

Hey @PickleRick, Yes, I've done this in production deployments. I think you're right re: line breaking, and aggQueue may be a better choice for re-processing timestamp extraction etc. before hitting typingQueue and transforms. My choice of parsingQueue was easier to justify relative to documentation and a typical Splunk administrators' high level understanding of Splunk internals after turnover. In a Splunk Cloud environment or an environment where I have an indexing tier but no mandatory heavy forwarding tier, I'd just use ingest actions / rulesets as you all did in this thread. I haven't observed problems with indexed terms. Here's a quick reconstruction with the iis source type (INDEXED_EXTRACTIONS = w3c for parsed events) that assumes we have authentication, authorization, connectivity, and network/host firewall configuration sorted: # Splunk Enterprise receiver ## indexes.conf [metatest] coldPath = $SPLUNK_DB/metatest/colddb homePath = $SPLUNK_DB/metatest/db maxTotalDataSizeMB = 750 thawedPath = $SPLUNK_DB/metatest/thaweddb ## inputs.conf [splunktcp://19997] disabled = 0 # send cooked data to parsingQueue instead of rulesetQueue route = has_key:_replicationBucketUUID:replicationQueue;has_key:_dstrx:typingQueue;has_key:_linebreaker:parsingQueue;absent_key:_linebreaker:parsingQueue ## props.conf [iis] TRANSFORMS-index = send_to_metatest ## transforms.conf [send_to_metatest] REGEX = . DEST_KEY = _MetaData:Index FORMAT = metatest ## restart Splunk # any forwarder ## C:\TEMP\u_ex260613.log #Software: Microsoft Internet Information Services 10.0 #Version: 1.0 #Date: 2026-06-13 14:00:00 #Fields: date time s-ip cs-method cs-uri-stem cs-uri-query s-port cs-username c-ip cs(User-Agent) sc-status sc-substatus sc-win32-status time-taken 2026-06-13 14:00:30 192.168.1.50 GET /index.htm - 443 - 192.168.1.51 Mozilla/5.0 200 0 0 15 ## outputs.conf [tcpout] defaultGroup = default-autolb-group [tcpout:default-autolb-group] server = splunk:19997 [tcpout-server://splunk:19997] ## restart splunkforwarder ## index the file (prompted for auth) "C:\Program Files\SplunkUniversalForwarder\bin\splunk.exe" add oneshot C:\TEMP\u_ex260613.log -index main -sourcetype iis # indexer ## roll hot buckets for walklex /opt/splunk/bin/splunk _internal call /data/indexes/walklex_test/roll-hot-buckets # search head ## index should contain one event index=metatest sourcetype=iis earliest=1 latest=now => 2026-06-13 14:00:30 192.168.1.50 GET /index.htm - 443 - 192.168.1.51 Mozilla/5.0 200 0 0 15 ## check terms and verify each term is indexed once | walklex type=fieldvalue index=metatest | table term count => term count c_ip::192.168.1.51 1 ...   ... View more

Hi @lorenzoromio, Expanding on @PickleRick's suggestions, you can brute force events through parsingQueue to open up not only typingQueue behavior but all functions you would expect a heavy forwarder or receiver to perform, irrespective of the event disposition. This is my preferred solution for handling events from an external forwarder in a pure Splunk environment. In an app or in $SPLUNK_HOME/etc/system/local/inputs.conf, override the [splunktcp] stanza route setting, and change has_key:_linebreaker:rulesetQueue to has_key:_linebreaker:parsingQueue: [splunktcp] route=has_key:_replicationBucketUUID:replicationQueue;has_key:_dstrx:typingQueue;has_key:_linebreaker:parsingQueue;absent_key:_linebreaker:parsingQueue You must still be mindful of the structure of received events, but all props.conf and transforms.conf settings will be available to you. Better practices using EVENT_BREAKER etc. at the forwarder do not change, but since the forwarder is outside your control, you'll have more flexibility in vetting and parsing events. ... View more

Here's an audio sample. First, update $SPLUNK_HOME/etc/system/local/web.conf to enable the <embed> tag. Splunk strips src attributes from <embed>, <audio>, etc. tags, but we can use <embed> with the following setting: [settings] dashboard_html_allow_embeddable_content = true Download this PCAP file: https://github.com/groktrev/junk/blob/74af4d0a1a35041b63df67019f2abc50aa3f085e/audio_sample.pcap.gz It contains a single session with a plaintext HTTP GET request and an MP3 file response. You can verify the file with, e.g., Wireshark. Index the file: $ gzip -d audio_sample.pcap.gz $ $SPLUNK_HOME/etc/apps/Splunk_TA_stream/linux_x86_64/bin/streamfwd -r audio_sample.pcap Create a Simple XML dashboard: <form version="1.1" theme="light"> <label>audio_player</label> <search id="all_audio_search"> <query> <![CDATA[ index=main sourcetype=stream:http mime_type=audio/* | stats latest(dest_content_hexadecimal) as dest_content_hexadecimal by site uri_path mime_type ]]> </query> <earliest>$time_tok.earliest$</earliest> <latest>$time_tok.latest$</latest> </search> <search base="all_audio_search" id="audio_search"> <query> <![CDATA[ | search uri_path=$uri_path_tok|s$ | fields mime_type dest_content_hexadecimal | lookup hexbase64lookup hexstring as dest_content_hexadecimal | eval audio_src="data:".mime_type.";base64,".base64string | fields audio_src mime_type ]]> </query> <done> <set token="audio_src_tok">$result.audio_src$</set> <set token="mime_type_tok">$result.mime_type$</set> </done> </search> <fieldset submitButton="false"> <input type="dropdown" token="uri_path_tok"> <label>URI Path</label> <fieldForLabel>uri_path</fieldForLabel> <fieldForValue>uri_path</fieldForValue> <search base="all_audio_search"></search> </input> <input type="time" token="time_tok" searchWhenChanged="true"> <label>Time Range</label> <default> <earliest>-24h@h</earliest> <latest>now</latest> </default> </input> </fieldset> <row> <panel depends="$uri_path_tok$"> <title>$uri_path_tok$</title> <html depends="$audio_src_tok$"><!-- <audio controls="controls"> <source src="$audio_src_tok$" type="$mime_type_tok$"/> </audio>--> <embed src="$audio_src_tok$" type="$mime_type_tok$" width="300" height="53"/> </html> </panel> </row> </form>  Make history! ... View more

While not as robust and functional as Cloudmeter's original products, which were focused on web site recording and replay, Splunk Stream scales well on bespoke hardware, e.g., AMD/Intel boxes with Napatech interfaces, and combined with Enterprise Security, provides functionality approximating but admittedly trailing Suricata and Zeek-based IDS and NDR solutions. Stream never found its footing outside the on-premises enterprise, and with the acquisition of SignalFx (also old news) and the merging of AppDynamics, I doubt it ever will. There's also this nugget in the 8.1.5 release notes: "As of version 8.1.5, Splunk Stream no longer ingests PCAP files." The functionality is still present, however. I built the toy example on 8.1.6. I wouldn't expect more than simple bug fixes at this point, and lack of support for HTTP/2 combined with widely adopted TLS forward secrecy make Stream moot for many distributed applications. ... View more

Hi @Hemant0808, Splunk Stream can ingest PCAP files. Splunk Stream's ability to index or hash payload content, including media files, varies by protocol. Splunk Stream is composed of three separate apps/add-ons: Splunk App for Stream https://splunkbase.splunk.com/app/1809 Splunk Add-on for Stream Wire Data https://splunkbase.splunk.com/app/5234 Splunk Add-on for Stream Forwarders https://splunkbase.splunk.com/app/5238 To install Splunk Stream in your environment, see: Splunk Cloud Platform https://help.splunk.com/en/splunk-cloud-platform/collect-stream-data/install-and-configure-splunk-stream/8.1/splunk-stream-architecture/splunk-stream-on-premise-deployment-architecture Splunk Enterprise https://help.splunk.com/en/splunk-enterprise/collect-stream-data/install-and-configure-splunk-stream/8.1/splunk-stream-architecture/splunk-stream-on-premise-deployment-architecture We'll use Splunk Enterprise as our baseline. HTTP is fully supported. Decryption of HTTPS (TLS) streams requires a captured RSA key exchange and the session's private key. Splunk Stream cannot decrypt TLS sessions with a Diffie–Hellman (DH) key exchange. Fields provided for HTTP file transfer streams are documented here: https://help.splunk.com/en/splunk-enterprise/collect-stream-data/install-and-configure-splunk-stream/8.1/protocols/file-transfer Enable the http stream. To configure streams to extract content, see: https://help.splunk.com/en/splunk-enterprise/collect-stream-data/use-splunk-stream/8.1/configure-streams/configure-streams-to-use-content-extraction Under the http stream, enable the following fields: dest_content dest_content_sha512_hash file_extracted_req file_extracted_resp file_size http_filename mime_type part_filename src_content src_content_sha512_hash Under the http stream, extract two new fields: Source Term: dest_content Name: dest_content_hexadecimal Description: dest_content_hexadecimal Extraction Type: Hexadecimal Source Term: src_content Name: src_content_hexadecimal Description: src_content_hexadecimal Extraction Type: Hexadecimal Save the changes. To index large payloads, we'll increase the [streamfwd] maxFieldSize from 10 KiB to 1 MiB. As a better practice, use a value large enough for your use cases but no larger. $SPLUNK_HOME/etc/apps/Splunk_TA_stream/local/streamfwd.conf [streamfwd] maxFieldSize = 1048576 We'll use a sample PCAP file from https://wiki.wireshark.org/SampleCaptures and the following reference: https://help.splunk.com/en/splunk-enterprise/collect-stream-data/install-and-configure-splunk-stream/8.0/configure-your-splunk-stream-installation/ingest-pcap-files Using the command line on Linux: [splunk@splunk ~]$ wget https://wiki.wireshark.org/uploads/__moin_import__/attachments/SampleCaptures/http_with_jpegs.cap.gz [splunk@splunk ~]$ gzip -d ~/http_with_jpegs.cap.gz [splunk@splunk ~]$ SPLUNK_HOME/etc/apps/Splunk_TA_stream/linux_x86_64/bin/streamfwd -r ~/http_with_jpegs.cap Ignore any Netflow or file mount point errors. In Splunk, search for the indexed content (note the 2004 dates from the PCAP file): index=main sourcetype=stream:http starttime=11/19/2004:00:00:00 endtime=11/20/2004:00:00:00 Notice that the dest_content and src_content fields contain the raw content bytes as JSON-escaped strings. These fields are useful for plaintext content like HTML, but they're a problem for binary content like images. Also notice that automatic JSON field extraction failed for the larger events. Under the http stream, disable the dest_content and src_content fields, save the configuration, and then re-index http_with_jpegs.cap as shown above. Search for the events again, but this time, add a mime_type search predicate: index=main sourcetype=stream:http starttime=11/19/2004:00:00:00 endtime=11/20/2004:00:00:00 mime_type=image/jpeg dest_content_hexadecimal contains the content bytes as a hexdecimal string. dest_content_sha512_hash contains the SHA-512 hash. uri_path contains the file name. Let's look at /Websidan/2004-07-SeaWorld/fullsize/DSC07858.JPG: index=main sourcetype=stream:http starttime=11/19/2004:00:00:00 endtime=11/20/2004:00:00:00 mime_type=image/jpeg uri_path=/Websidan/2004-07-SeaWorld/fullsize/DSC07858.JPG | fields mime_type dest_content_hexadecimal | head 1 Can we do anything interesting with the content? Yes, we can! While we can manipulate hex strings in Splunk, we will eventually have problems mapping bytes to UTF-8, and we'll need to implement Base64 encoding in either SPL or an external script. To work around these problems, we'll create a simple external lookup script to encode hex strings as Base64. As a better practice, create an app directory under $SPLUNK_HOME/etc/apps: $ mkdir -p $SPLUNK_HOME/etc/apps/hexbase64lookup/bin $SPLUNK_HOME/etc/apps/hexbase64lookup/default $SPLUNK_HOME/etc/apps/hexbase64lookup/metadata Then create the following files: $SPLUNK_HOME/etc/apps/hexbase64lookup/bin/hexbase64lookup.py #!/usr/bin/env python import base64 import csv import sys def main(): if len(sys.argv) != 3: print('Usage: python base64lookup.py hexstring base64string') sys.exit(1) hexstringfield = sys.argv[1] base64stringfield = sys.argv[2] sys.stdin.reconfigure(errors='surrogateescape') sys.stdout.reconfigure(errors='surrogateescape') csv.field_size_limit(sys.maxsize) infile = sys.stdin outfile = sys.stdout r = csv.DictReader(infile) header = r.fieldnames w = csv.DictWriter(outfile, fieldnames=r.fieldnames) w.writeheader() for result in r: if result[hexstringfield] and result[base64stringfield]: w.writerow(result) elif result[hexstringfield]: hexstring = result[hexstringfield] hexstringbytes = bytes.fromhex(hexstring) base64bytes = base64.standard_b64encode(hexstringbytes) base64string = base64bytes.decode('utf-8', errors='surrogateescape') result[base64stringfield] = base64string w.writerow(result) elif result[base64stringfield]: base64string = result[base64stringfield] base64bytes = base64string.encode('utf-8', errors='surrogateescape') hexstringbytes = base64.standard_b64decode(base64bytes) hexstring = hexstringbytes.hex() result[hexstringfield] = hexstring w.writerow(result) main()  $SPLUNK_HOME/etc/apps/hexbase64lookup/default/transforms.conf [hexbase64lookup] external_cmd = hexbase64lookup.py hexstring base64string fields_list = hexstring,base64string $SPLUNK_HOME/etc/apps/hexbase64lookup/metadata/default.meta [searchscripts/hexbase64lookup.py] access = read : [ * ], write : [ admin ] export = system [transforms/hexbase64lookup] access = read : [ * ], write : [ admin ] export = system Restart Splunk! Now we can get the Base64 representation of the image data, in this case as <img> src attribute data: index=main sourcetype=stream:http starttime=11/19/2004:00:00:00 endtime=11/20/2004:00:00:00 mime_type=image/jpeg uri_path=/Websidan/2004-07-SeaWorld/fullsize/DSC07858.JPG | head 1 | fields mime_type dest_content_hexadecimal | lookup hexbase64lookup hexstring as dest_content_hexadecimal | eval img_src="data:".mime_type.";base64,".base64string Now what? Let's create a Simple XML dashboard to view the image! Here's the source: <dashboard version="1.1" theme="light"> <label>image_viewer</label> <search id="image_search"> <query><![CDATA[ index=main sourcetype=stream:http starttime=11/19/2004:00:00:00 endtime=11/20/2004:00:00:00 mime_type=image/jpeg uri_path=/Websidan/2004-07-SeaWorld/fullsize/DSC07858.JPG | head 1 | fields mime_type dest_content_hexadecimal | lookup hexbase64lookup hexstring as dest_content_hexadecimal | eval img_src="data:".mime_type.";base64,".base64string ]]></query> <done> <set token="img_src_tok">$result.img_src$</set> </done> </search> <row> <panel> <html> <img src="$img_src_tok$"/> </html> </panel> </row> </dashboard> And here's a screenshot (at 50%): Dolphins! To display image content in a Dashboard Studio dashboard, we can store the data in a KV store collection used by Dashboard Studio (not shown). Will this work for other protocols and content types? It might! You've labeled the question "HTTP Event Collector," which is a specific Splunk feature unrelated to indexing PCAP data. Let me know which protocol you're trying to extract media and audio files from, and we can find a sample PCAP file and adapt the solution. Disclaimer: Do not use the methods described above for security research in production. Use an appropriate sandbox. ... View more