HOW TO: Connect Thunderbolt devices for best performances

With the 2019 Mac Pro

The 2019 Mac Pro can have different Thunderbolt port and controllers (or BUS) numbers. Indeed some of the Graphics cards can add more Thunderbolt ports and controllers.

Note that we indicate BUS numbers in our screenshots below, but these may vary depending on your configuration. But what's important is not the BUS number, but to know that the Thunderbolt ports go by pairs, as there are 2 ports per BUS.

 

"Built-in" Thunderbolt ports:

If you don't consider the graphics cards, the Mac Pro comes with 4 Thunderbolt ports:

  • 2 connectors (1 BUS) at the back of the Mac Pro (these ports are on a PCIe card so they could in theory be removed as well):
  • MP7_BackTop_TB_BUS2_SMALL.png
  • 2 connectors (1 BUS) at the top of the Mac Pro (these are the only Thunderbolt ports that can not be removed and will always be there):
  • MP7_Top_TB_BUS3_SMALL.png

 

Thunderbolt ports on Graphics cards

Some of the graphics cards that can be purchased with the Mac Pro can add Thunderbolt ports. These can be used either to connect Thunderbolt displays, or any Thunderbolt device.

  • AMD Radeon Pro 580X: No external Thunderbolt ports
  • AMD Radeon Pro W5700X:  4 thunderbolt ports (2 BUS)
  • AMD Radeon Pro Vega II:  4 thunderbolt ports (2 BUS)
  • AMD Radeon Pro Vega II Duo:  4 thunderbolt ports (2 BUS)

For all the cards that have Thunderbolt ports, the BUS are grouped as follows:

MP7_BackBottomVegaII_TB_BUS0-1_SMALL.png

 

About PCIe usage and "Expansion Slot Utility"

It is important to note that Thunderbolt uses PCIe bandwidth. Each Thunderbolt to which at least one device is connected will allocate 4 lanes of PCIe3. 

And thus it is important to check the "Expansion Slot Utility" every time you connect a new device, in order to make sure that it will have enough bandwidth. For example:

  • if you have too many PCIe cards inside the Mac, you may not have enough PCIe bandwidth to drive the additional M|Family device.
  • if you connect too many Thunderbolt devices on your Mac (specially if you have 2 x Vega II (Duo), which add 8 connectors (4 BUS)). That would make a total of 6 BUS, using each 4-lane of PCIe 3.0...

Note that Thunderbolt devices will only use Pool B, so if you do a manual configuration, make sure to leave a little room on Pool B for your Thunderbolt devices.

MP7_ExpansionSlotUtility_ThunderboltDevices.png

For more information on how to handle PCIe connections on your Mac Pro, check the following article.

 

A few facts

  • each Thunderbolt BUS where a device is connected will allocate 4 lanes of PCIe3
  • all Thunderbolt BUS are allocated to PCIe pool B only. As a pool has 16 lanes, it means that you can use a maximum of 4 BUS before starting to share the bandwidth with another.
  • even if the PCIe expansion utility shows that you are over 100% of allocation, it does not mean necessarily that you'll have issues. You may have issues if all Thunderbolt BUS work at their highest speed at the same time. So it is best to avoid this, but check what each one does.
  • the Apple I/O card in slot 8 already allocates 4 lanes of pool B, even if nothing is connected to it. Connecting a Thunderbolt device to it will allocate 4 more lanes, but in fact if you did not connect anything in the USB ports, it will only use 4 lanes, not 8.
  • when using a Thunderbolt 3 to Thunderbolt 2 adapter, 4 lanes of PCIe will be allocated, even if you only connected a display using Display Port bandwidth, and not PCIe. That's because the adapter has a Thunderbolt controller, and the macOS can not know in advance what kind of devices will be connected. It can only know what in theory could go through that port.

 

A few recommendations 

  • each device that uses PCIe should be connected on its own BUS
  • a Thunderbolt monitor (that only uses Display Port) should be connected on the Thunderbolt ports of the graphics card so the "path" is simpler.

 

Recommended configuration for the M|Family

As the M|Family includes a standard PCIe card, we recommend connecting the card inside the Mac Pro in one of its PCIe slots.

If you don't have any PCIe slot available, you can connect it (if you have one of the models with a Thunderbolt box), to one of the Thunderbolt 3 connector. For best results, we recommend that  no other device is connected to the same BUS.

You should prefer the built-in ports (on the Mac Io card) over the ones on the Graphics card. Make sure also to check the "Expansion Slot Utility".

 

 

 

With the 2018 Mac mini

The Mac mini comes with 4 Thunderbolt ports, and 2 controllers (or BUS) which are mapped as follows:

Macmini2018_Thunderbolt_BUSmapping.png

Recommended configuration for the M|Family

You can plug in your M|Family device to any of the 2 Thunderbolt Controller/BUS. But for best results, you should not connect anything else on the same Thunderbolt BUS as the one where your M|Family device is connected. 

 

 

With the 2017 iMac Pro

The iMac Pro comes with 4 Thunderbolt ports, and 2 controllers (or BUS) which are mapped as follows:

image.png

© Apple Inc.

 

Recommended configuration for the M|Family

You can plug in your M|Family device to any of the 2 Thunderbolt Controller/BUS. But for best results, you should not connect anything else on the same Thunderbolt BUS as the one where your M|Family device is connected. 

 

 

With the 2013 Mac Pro

The 2013 Mac Pro has six (6) Thunderbolt 2 ports that you can use. Below are some useful tips to know.

 

Choosing the right Thunderbolt port / BUS

The 2013 Mac Pro has six Thunderbolt ports but "only" 3 busses, which are as follows:

HT5918-macpro-multipledisplay_ports-001-en.png

© Apple Inc.

 

You should connect devices that use a lot of bandwidth on different Thunderbolt busses. Also if you are using the built-in HDMI output, prefer to use busses 1 & 2 as the built-in HDMI output is also on Thunderbolt Bus 0.

 

Recommended configuration for the M|Family

The M|80, M|62 and M|44 use a lot of bandwidth, so you should connect your M|Family device alone on one BUS, he should not share the bandwidth with another device. See the diagram below of one recommendation.

 

Of course, this is one possible configuration, and you could connect the M|family device on BUS 1 and your storage on BUS 2, that would make no difference. But avoid BUS 0 that can share the bandwidth with the Display.

 

 

 

 

How many channels can go through one Thunderbolt ?

When we receive the video frames from the video card, we receive them as uncompressed. So the available bandwidth in the Thunderbolt is really important as it will tell how many channels of uncompressed video we can receive through one Thunderbolt cable and it depend on a lot of variables. Among which:

  • if using a PCIe card, it will depend on if the card is a PCIe gen 2 or gen 3, and the number of lanes it uses.
  • the device and expansion chassis itself.

Differences between PCIe gen 2 and gen 3

For example with the M|Family products that use PCIe gen 2 card, we can reach maximum:

  • in Thunderbolt 2 (or 3) with the Sonnet Echo Express SEL: 1.200MB/s
  • in Thunderbolt 3 with the Sonnet Echo Express SEIII: 2.100MB/s

And with the M|Family products that use PCIe gen 3 card (such as the Deltacast 12G-2C), we can reach  2.100MB/s with both the Thunderbolt 3 models Sonnet Echo Express SEL and SEIII.

Calculating the number of uncompressed frames that can go through

Knowing this, and how to calculate how much data an uncompressed frame uses, you can make calculations of how many channels of uncompressed video can go through the Thunderbolt. Here are a few examples.

First with the Deltacast cards in the Echo Express SEL (Thunderbolt 2):

  • 1080p25 -> 8bit = 11 channels   /  10bit = 8 channels
  • 1080p30 -> 8bit = 9 channels   /  10bit = 7 channels
  • UHDp25 -> 8bit = 2 channels   /  10bit = 1 channel
  • UHDp30 -> 8bit = 2 channels   /  10bit = 1 channel

And with the PCIe gen2 cards in the Echo Express SEIII (Thunderbolt 3), or with the PCIe gen 3 cards in both Thunderbolt 3 models:

  • 1080p25 -> 8bit = 20 channels   /  10bit = 15 channels
  • 1080p30 -> 8bit = 17 channels   /  10bit = 12 channels
  • 1080p50 -> 8bit = 10 channels   /  10bit = 7 channels
  • 1080p60 -> 8bit = 8 channels   /  10bit = 6 channels
  • UHDp25 -> 8bit = 5 channels   /  10bit = 3 channels
  • UHDp30 -> 8bit = 4 channels   /  10bit = 3 channels
  • UHDp50 -> 8bit = 2 channels   /  10bit = 1 channel
  • UHDp60 -> 8bit = 2 channels   /  10bit = 1 channel

 Note that:

  • This only tells how many channels you can "receive" on your Mac using one single device and Thunderbolt cable. You could receive more if you use multiple Thunderbolt devices, connected to different controllers/BUS on your Mac.
  • This does not tell you how many channels you will be able to encode, which depend on your CPU and GPU.
  • Of course you will need the video card that support
  • These are calculated theoretical values, and we advise that you test in your workflow as results can vary.
  • On the 2019 Mac Pro, the PCIe bandwidth is shared between the internal PCIe slots, and the Thunderbolt ports.

 

About daisy-chain

We do not advise that you daisy chain anything behind your M|Family device as it will have to share the bandwidth with other device, and we already use a lot. However if you lack Thunderbolt connectors and really have to, you could try and consider this:

  • Apple recommends that you place the device that ask the most bandwidth first and then "smaller" devices after.
  • In theory the Display Port signal is separated from the PCIe signal, so you could connect a monitor after, but prefer not to use a monitor with too much resolution, so it does not use too much bandwidth still.
  • Also, the input and output bandwidth are separated, so again in theory you could connect a storage device if you only write to it, but not read. (so it does not use the "input" bandwidth that the M|Family needs).

 

What happens if you reach the limit ?

There will be 2 consequences:

  • In the Activity and Logs window, you will see a log saying: "Video Card buffer underrun". This is because we can not get the frames quick enough from the video card.
  • As we can't retrieve the frames quick enough from the video card, we will duplicate the corresponding video frames and drop audio, so there will be drops in both audio and video in the recorded file.
Have more questions? Submit a request

Comments

  • Avatar
    Robin Erard

    I have an issue about this PCI Pool selector.

    I have two Vega II DUO, so 4 BUS on them and 2 BUS with Mac I/O and top ports. So 6 Thunderbolt 3 BUS.

    I have 3 RAID HDD tower (all of them in Thunderbolt 2) and a Blackmagic 4 k extreme (connected by Thunderbolt 2 also), + 2k monitor (connected in Display port with an Thunderbolt->Display Port adapter.)

    If I connect my material as mentioned bellow. Pool B goes up to 150%... and POOL A stays at 0%, but I don't understand why.

    - Vega II Duo Number 1 BUS 0 : RAID tower Number 1
    - Vega II Duo Number 1 BUS 1 : RAID tower Number 2
    - Vega II Duo Number 2 BUS 0 : RAID tower Number 3
    - Vega II Duo Number 2 BUS 1 : Monitor 2k
    - Mac I/O : Blackmagic 4k extrem

    As I understand, everything should be right... but my POOL B exceed 100% and I can't force the Mac Pro to use POOL A... Do you know why ? Are Thunderbolt material automatically assigned to POOL B ?

    I made an other test. If I connect my material as mentioned bellow. Pool B don't exceed 100%... and POOL A stays at 0%.

    - Vega II Duo Number 1 BUS 0 : monitor 2k
    - Vega II Duo Number 2 BUS 0 : RAID tower Number 1
    - Mac I/O : RAID tower Number 2
    - Mac I/O : RAID tower Number 3
    - Top Thunderbolt PORT : Blackmagic 4k extrem

    For me it's not normal. And more strange, if I disconnect everything PCI indicator tells me that PCI cards aren't arranged in optimal configuration. (But this configuration comes from APPLE, two MPX and Mac I/O card).

    Maybe, I'm wrong on something, but I don't know about what.

    Best

    Robin