I'd like to add that this answer is half correct. The Indexers RECEIVE data on port 9997, but the Universal Forwarder and Heavy Forwarder SEND the data over a random port.
Within the packet structure, the SRC will be the [ForwarderIP]:[random port] and the DSC will be [Indexer IP]:9997
I would also open port 8089 from yoru Search Heads or Indexers to UF, HF, or LFs. (Indexer, Search Head, UF, HF, or LF) If you plan on using the CLI. Also this port is required for the (default conf) Deployment Server.