NetFPGA SUME Reference Manual

NetFPGA-SUME

Overview

Powered by Xilinx's Virtex-7 XC7V690T FPGA, the NetFPGA-SUME is an ideal platform for high-performance and high-density networking design.

32 GTH serial transceivers have been used to provide access to 8 lanes of end-point PCI-E (Gen3 x8), 4 SFP+ (10Gbps) ports, 2 SATA-III ports (6Gbps), and 18 data-rate-adjustable GTH ports through a HPC-FMC connector and a QTH connector.

Wide high-speed memory interfaces in the form of three 72 MBit QDRII+ SRAMs with 36 bit buses and two 4GB DDR3 SODIMMs with 64 bit buses provide an ideal memory solution for common networking applications.

The NetFPGA SUME is compatible with Xilinx’s new high-performance Vivado® Design Suite as well as the ISE® toolset, which includes ChipScope™ and EDK. The Virtex-7 XC7V690T FPGA is not a WebPack device, which means full licenses will need to be acquired for these tools in order to build designs that target the NetFPGA SUME. Licensing information for Vivado can be found here. Academic institutes can make a request to the Xilinx University Program for a donation of full Vivado licenses here.

A simplified block diagram that depicts the major features of the NetFPGA SUME is seen in figure 1.

NetFPGA SUME Block Diagram Figure 1. NetFPGA SUME Block diagram.

Feature List

  • FPGA
    • Xilinx Virtex-7 XC7V690T FFG1761-3
  • Memory
    • Two 4GB DDR3 SODIMMs (MT8KTF51264Hz-1G9E1)
      • 64 bit wide buses clocked at 850 MHz (1700 mbps)
    • Three 72Mbit QDRII+ SRAMs (CY7C25652KV18-500BZXC)
      • 36 bit wide buses clocked at 500 MHz (1000 mbps)
  • Storage
    • Two SATA III ports (6 gbps via GTH transceivers)
    • Micro-SD Card Slot
    • Two 512Mbit Micron StrataFlash parallel flash modules (PC28F512G18A)
      • For bitfile storage only
  • Communication Interfaces
    • PCI-E Gen3 x8 supporting 8Gbps/lane
    • Four SFP+ interfaces supporting 10Gbps
      • Four globally unique MAC addresses
    • USB-UART
    • I2C Pmod Port
  • Expansion Connectors
    • QTH Connector (8 GTH transceivers)
    • One HPC FMC Connector (10 GTH transceivers and 68 User I/Os)
    • One 12-pin Pmod Port (8 User I/Os)
  • Programming
    • Micro USB Connector for JTAG programming and debugging (shared with USB-UART interface)
    • Xilinx CPLD XC2C512 for FPGA configuration from parallel flash
  • Power Management
    • Two Linear Technology Power System Managers (LTC2974)
      • Provide current measuring on all major power rails
  • Other Features
    • User LEDs and Push Buttons
    • PROG Push Button for manual FPGA Reset
    • FPGA Configuration LEDs
    • I2C Mux (PCA9548A) for controlling all onboard I2C buses

NetFPGA-SUME Component Callouts

NetFPGA Organization

This board is supported by reference designs and IP created by the NetFPGA organization.

For more information on the NetFPGA organization, go here.

Functional Description

Power

Input Supply

The NetFPGA-SUME receives power via a 2×4 pin PCI Express Auxiliary Power Connector. The 2×4 pin PCI Express Auxiliary Power receptacle (header J14) can accept both 2×3 and 2×4 pin PCI Express Auxiliary Power Plugs found on a standard ATX power supply. When installed on a PC motherboard, you can plug the 2×3 or 2×4 pin PCI Express power supply connector directly into header J14. When used in standalone mode (without a PC motherboard), pins 15 and 16 of the main 20 pin connector of the standard ATX power supply must be shorted together, as shown in Fig. 2. If these pins are not shorted together, then the power supply will not turn on.

 Pin 15 and 16 of Standard ATX power supply shorted together Figure 2. Pin 15 and 16 of Standard ATX power supply shorted together.

According to Revision 1.0 of the PCI Express 225 W/300 W High Power Card Electromechanical Specification, the 2×3 pin plug is guaranteed to deliver up to 75 Watts of power, while the 2×4 pin plug is guaranteed to deliver up to 150 Watts of power. While the board may be powered by either a 2×3 pin or a 2×4 pin PCI Express Auxiliary Power plug, due to the potential for high power consumption, Digilent recommends using a 2×4 pin plug to provide power whenever possible.

 Power Connector (J14) Figure 3. Power Connector (J14).

Figure 3 describes pin-out of the power connector (header J14) when a 2×4 pin or a 2×3 pin plug is used. The Sense0 and Sense1 pins are to be connected to GND when power is present, and left floating otherwise. Since the 2×3 pin plug does not include a Sense1 pin, it's possible to determine what type of plug is present, and thus how much power can be consumed.

The FPGA logic can determine whether or not a 2×4 pin is present by enabling an internal pull-up on pin AW42 and then checking the state of that pin. If logic '0' is seen on AW42, then a 2×4 plug is connected and up to 150 Watts of power can be drawn. If logic '1' is seen on AW42, then a 2×3 plug is connected, and the board's power consumption should be limited to 75 Watts or less.

Power Supply Topology

The high performance Virtex-7 FPGA, QDRII+ memories, and DDR3 memories featured on the NetFPGA-SUME require several different supply voltages (supply rails) in order to function. These components also require that the supply rails are sequenced on and off in a particular order. Table 1 lists the various supply rails, their nominal voltages, and rated output currents.

Supply Rail Nominal Voltage Rated Output Current
VCC1V0 1.0V 40A
VCC1V8 1.8V 15A
VCC2V0 2.0V 2A
MGTAVCC 1.0V 8A
MGTAVTT 1.2V 3A
VCC3V3 3.3V 15A
VCC1V5 1.5V 15A
MGTVAUX 1.8V 1A
DDRVTT 0.75V +/- 3A
QDRVTT 0.75V +/- 3A

Table 1. Supply rails, voltages, and currents.

These supply rails are derived from the 12V input (VCC12V0, comes from header J14) using eight high efficiency switching regulators and one low drop out (LDO) linear regulator from Linear Technology. Since both the DDR3 and QDRII+ I/O supplies are powered from the VCC1V5 rail, two ferrite beads are also included to prevent high speed switching noise caused by one memory from affecting the other. Figure 5 shows how the various supplies are derived from the input.

A Linear Technology LTC6909 is used to generate six out of phase 302 KHz clocks. Each clock is 60 degrees out of phase with any of the other clock outputs (see Fig. 4). These out of phase clocks are used as the input clocks for the regulators that produce the high power output supply rails (VCC1V0, VCC1V5, VCC1V8, VCC3V3, and MGTAVCC). The LTC3839, which produces the VCC1V0 supply rail, is a dual phase converter that directly utilizes the OUT1 clock and indirectly utilizes the OUT4 clock. The use of out of phase clocks reduces the input RMS ripple current.

LTC6909 Clock Output Phase Relationship Figure 4. LTC6909 Clock Output Phase Relationship.

Regulator Topology Figure 5. Regulator Topology.

Power Supply Sequencing and Supervising

The components on the NetFPGA-SUME require that the supply voltages be sequenced on and off in a particular order. The NetFPGA-SUME utilizes two Linear Technology LTC2974s to ensure that these sequencing requirements are met. Each LTC2974 supports “cascade sequence ON with time-based sequence off” and can monitor the input voltage, four output voltages, four output currents, and four external temperatures using a 16-bit ADC. Additionally, the LTC2974 can margin and trim up to four output voltages using a 10-bit DAC, allowing for more precise output voltages.

LTC2974 Sequencer and Supervisor Figure 6. LTC2974 Sequencer and Supervisor.

Figure 6 depicts the connections between the two LTC2974s, as well as the signals that are used to control the power on and off sequence. When the input voltage (VCC12V0) exceeds 10 volts, the LTC2974s will perform a power on sequence when the power switch (SW1) is placed in the “ON” position. When a power on sequence is performed, the rails come up in the following order:

  1. VCC1V0
  2. VCC1V8
  3. VCC2V0
  4. MGTAVCC
  5. MGTAVTT
  6. VCC3V3
  7. VCC1V5, QDRVTT, and DDRVTT
  8. MGTVAUX

When the input voltage falls below 9 volts, or the power switch transitions to the “OFF” position, the LTC2974s perform a time-based off sequence and the rails come down in the following order:

  1. MGTVAUX
  2. VCC1V5, QDRVTT, and DDRVTT
  3. VCC3V3
  4. MGTAVTT
  5. MGTAVCC
  6. VCC2V0
  7. VCC1V8
  8. VCC1V0

Power ON/OFF Sequence Figure 7. Power ON/OFF Sequence.

The LTC2974s constantly monitor the output voltage, current, and temperature associated with each channel (supply rail). This information, referred to as telemetry data, is used to determine the on status of each supply rail, as well as monitor for fault and warning conditions. When a fault or a warning occurs, the FPGA application may be notified via an interrupt that's signaled by the LTC2974’s ALERTB, AUXFAULTB, or FAULTB1 pins. The FPGA application may then read (using I2C) one or more of the LTC2974 status registers (defined in the datasheet) to determine the source of the fault or the warning. The output voltage, current, power, and temperature associated with any channel may also be read using the applicable PMBUS (I2C) commands, which are defined in the LTC2974 datasheet.

In order to generate faults and warnings, each channel of the LTC2974 must be configured with a nominal output voltage, under voltage warning limit, over voltage fault limit, under current warning limit, over current warning limit, over current fault limit, under current fault limit, under temperature warning limit, under temperature fault limit, and over temperature fault limit. Tables 2 & 3 describe the voltage and current limits as preconfigured by Digilent during the manufacturing process.

Supply Rail Under Voltage Fault Limit Under Voltage Warning Limit Nominal Voltage Overvoltage Warning Limit Overvoltage Fault Limit
VCC1V0 0.9V 0.925V 1.0V 1.075V 1.1V
VCC1V8 1.62V 1.665V 1.8V 1.935V 1.98V
VCC2V0 1.8V 1.85V 2.0V 2.15V 2.2V
MGTAVCC 0.9V 0.925V 1.0V 1.075V 1.1V
MGTAVTT 1.08V 1.11V 1.2V 1.29V 1.32V
VCC3V3 2.97V 3.0525V 3.3V 3.5475V 3.63V
VCC1V5 1.35V 1.3875V 1.5V 1.6125V 1.65V
MGTVAUX 1.62V 1.665V 1.8V 1.935V 1.98V
DDRVTT N/A N/A 0.75V N/A N/A
QDRVTT N/A N/A 0.75V N/A N/A

Table 2. Voltage fault and warning limits.

Note: the DDRVTT and QDRVTT rails are not directly monitored by the LTC2974s.

Supply Rail Undercurrent Fault Limit Rated Current Overcurrent Warning Limit Overcurrent Fault Limit
VCC1V0 -1A 40A 40A 44A
VCC1V8 -2A 15A 15A 15.5A
VCC2V0 -1A 2A 2A 3A
MGTAVCC -1A 8A 8A 8.5A
MGTAVTT -1A 3A 3A 3.1992A
VCC3V3 -1A 15A 15A 15.5A
VCC1V5 -1A 15A 15A 15.5A
MGTVAUX -1A 1A 1A 1.0996A
DDRVTT N/A 3A N/A N/A
QDRVTT N/A 3A N/A N/A

Table 3. Current fault and warning limits.

Note: the DDRVTT and QDRVTT rails are not directly monitored by the LTC2974s.

Fault and Warning Interrupt Sources

When the LTC2974 detects a fault or a warning condition it may signal an interrupt by driving the ALERTB, AUXFAULTB, or FAULTB1 pins.

The ALERTB pin of the LTC2974 is an open drain output that is driven low whenever a fault or warning occurs. The ALERTB pins of the two LTC2974s (IC43 and IC44) are connected in a wire-and fashion via the PCON_ALERT_B net to pin J41 of the FPGA (IC12), as shown in Fig. 7. Enabling the internal pull-up on pin J41 will allow the FPGA application to use this pin as an interrupt when any of the following situations occur:

  • Output overvoltage or under voltage fault/warning
  • Output over current or under current fault/warning
  • Over temperature fault/warning
  • Channel output voltage has not reached or exceeded the under voltage fault limit set for that channel within TON_MAX_FAULT_LIMIT milliseconds (set to 15ms) of the output being enabled

When any of the above faults occur the PCON_ALERT_B net will be driven low until the fault condition has been removed and the CLEAR_FAULTS command has been sent to both of the LTC2974s (IC43 and IC44).

ALERTB Interrupt Source Figure 8. ALERTB Interrupt Source.

The AUXFAULTB pins of the two LTC2974s are connected in a wire-and fashion to the cathode of a Shottky diode via the AUXFAULT net as shown in Fig. 9. The anode of the Shottky diode is connected to pin M41 on the FPGA via the PCON_AUXFAULT_B net and protects the FPGA pin from high voltages. The LTC2974s are configured to drive the AUXFAULTB pin low when any of the follow conditions occur:

  • Output overvoltage fault on any channel
  • Output over current or under current fault on any channel

When any of the above conditions occurs the AUXFAULT net will be driven low, which causes the PCON_AUXFAULT_B net to be pulled low through the diode. The AUXFAULT net will remain driven low until LTC2974 experiencing the fault condition is command to re-enter the ON state. Enabling the internal pull-up on pin M41 will allow the FPGA application to detect when one of the faults described above has occurred.

AUXFAULTB Interrupt Source Figure 9. AUXFAULTB interrupt source.

The FAULTB1 pin of the LTC2974 is bi-directional open-drain input/output that can be configured to drive low in response to any channel entering a “faulted off state”. The LTC2974 can also be configured to disable any given channel in response to a logic low being detected on the FAULTB1 pin. However, it has been preconfigured by Digilent during manufacturing to serve strictly as an output that indicates when any channel has faulted off.

The FAULTB1 pin of the two LTC2974s are connected in a wire-and fashion to the gate of a transistor (N-FET). This transistor connects to the PCON_FAULT1 net, which is in turn connected to pin N40 of the FPGA, as shown in Fig. 10. When neither of the FAULTB1 pins is asserted low, the gate of the transistor is pulled high and the PCON_FAULT1 net is connected to ground. When any channel enters the “faulted off state,” the gate of the transistor is driven low and the transistor turns off. Enabling an internal pull-up on pin N40 will allow the FPGA application to detect logic '1' when any channel has faulted off and logic '0' when no channels have faulted off.

FAULTB1 Interrupt Source Figure 10. FAULTB1 interrupt source.

Power Consumption

The power consumed by the NetFPGA-SUME is largely dependent on the number of resources utilized by a given design, and the complexity of that design. This makes estimating power consumption for any particular application difficult. However, during the design process it was necessary to estimate the worst case scenario power consumption for each of the supply rails and to come up with a total power budget. Table 4 lists the supply voltages, their rated output current, and the maximum power that they can output.

Supply Rail Nominal Voltage Rated Current Maximum Power Output
VCC1V0 1.0V 40A 40W
VCC1V5 1.5V 15A 22.5W
VCC1V8 1.8V 15A 27W
VCC2V0 2.0V 2A 4W
VCC3V3 3.3V 15A 49.5W
MGTAVCC 1.0V 8A 8W
MGTAVTT 1.2V 3A 3.6W
VCC12V0 (FMC) 12.0V 1A 12W
VCC12V0 (FPGA FAN) 12.0V 50mA 0.6W

Table 4. Supply rails and their maximum power output.

Note: The output power of the DDRVTT, MGTVAUX, and QDRVTT rails are not listed because these supply rails are derived from the VCC3V3 rail.

Maximum Output Power: 40+22.5+27+4+49.5+8+3.6+12+0.6=167.2 Watts

The actual power consumed from the input supply is greater than maximum output power due to the switching regulators not being 100% efficient. The switching regulators used on the NetFPGA-SUME were designed to operate at approximately 90% efficiency. The total power consumed from the input supply can be as high as ((40+22.5+47+4+49.5+8+3.6)/0.90)+12+0.6=184.4 Watts. However, this number can be a bit misleading as it assumes that a mezzanine module is attached to the FMC connector and is drawing the maximum allowable current from the VADJ, 3P3V, and 12P0V rails simultaneously.

According to the FMC specification, which is defined by ANSI/VITA 57.1, an FMC carrier card (NetFPGA-SUME) must be capable of supplying 4A to VADJ (1.8V, VCC1V8), 3A to 3P3V (3.3V, VCC3V3), and 1A to 12P0V (12.0V, VCC12V0). This implies a total input power consumption of ((4*1.8)+(3*3.3))/0.9+(1*12)=29.1 Watts by the FMC mezzanine module. However, ANSI/VITA 57.1 says that mezzanine modules may dissipate a maximum of 10 Watts. This means that the maximum output power is actually 167.2-(29.1-10)=148.1 Watts.

Determining the input power consumption after taking into account the 10 Watt output power limitation of the FMC connector is difficult, as we do not know which supply rails will be utilized by an attached FMC mezzanine module. Assuming that all power was consumed from VADJ and 3P3V, the total input power consumption would be reduced by 12+((19.1-12)/0.9)=19.9 Watts. As a result, the input power consumption could be as high as 184.4-19.9=164.5 Watts when an FMC mezzanine module is attached. If no FMC mezzanine module is attached then the maximum input power consumption is (40+22.5+(1.8*11)+(3.3*12)+8+3.6)/0.9+0.6=153.4 Watts.

FPGA Configuration

After power-on, the Virtex-7 FPGA must be configured (or programmed) before it can perform any functions. You can configure the FPGA in one of two ways:

  1. A PC can use the Digilent USB-JTAG circuitry (port J16, labeled “PROG”) to program the FPGA any time the power is on.
  2. One of four bitstream files stored in the parallel flash can be loaded by the onboard CPLD.

 NetFPGA-SUME Configuration Options

The figure above shows the different options available for configuring the FPGA. An on-board “mode” jumper (JP1) selects between the two programming modes.

The FPGA configuration data is stored in files called bitstreams that have the .bit file extension. The ISE or Vivado software from Xilinx can create bitstreams from VHDL, Verilog®, or schematic-based source files (in the ISE toolset, EDK is used for MicroBlaze™ embedded processor-based designs).

Bitstreams are stored in SRAM-based memory cells within the FPGA. This data defines the FPGA’s logic functions and circuit connections, and it remains valid until it is erased by removing board power, by pressing the reset button attached to the PROG input, by writing a new configuration file using the JTAG port, or by triggering the onboard CPLD to load a new bitstream from the parallel flash.

A Virtex-7 690T bitstream is typically 229,878,496 bits and can take a long time to transfer. The time it takes to program the NetFPGA-SUME can be decreased by compressing the bitstream before programming, and then allowing the FPGA to decompress the bitstream itself during configuration. Depending on design complexity, compression ratios of 10x can be achieved. Bitstream compression can be enabled within the Xilinx tools (ISE or Vivado) to occur during generation. For instructions on how to do this, consult the Xilinx documentation for the toolset being used.

After being successfully programmed, the FPGA will illuminate the “DONE” LED. Pressing the “PROG” button at any time will reset the configuration memory in the FPGA. After being reset, the FPGA will immediately attempt to reprogram itself from the parallel flash, assuming JP1 is not loaded.

The following sections provide greater detail about programming the NetFPGA-SUME using the different methods available.

JTAG Configuration

The Xilinx tools typically communicate with FPGAs using the Test Access Port and Boundary-Scan Architecture, commonly referred to as JTAG. During JTAG programming, a .bit file is transferred from the PC to the FPGA using the onboard Digilent USB-JTAG circuitry (port J16) or an external JTAG programmer, such as the Digilent JTAG-HS2, attached to port J9. You can perform JTAG programming any time after the NetFPGA-SUME has been powered on, regardless of whether or not the mode jumper (JP1) is set. If the FPGA is already configured, then the existing configuration is overwritten with the bitstream being transmitted over JTAG. Setting the mode jumper is useful to prevent the FPGA from being configured from the parallel flash.

Programming the NetFPGA-SUME with an uncompressed bitstream using the on-board USB-JTAG circuitry usually takes around a minute. JTAG programming can be done using the hardware server in Vivado or the iMPACT tool included with ISE.

Configuration using Parallel Flash

In order to meet the PCIe specification, an expansion card must be able to respond to PCI enumeration commands within 200 milliseconds of the power supplies becoming stable. On the NetFPGA-SUME, responding to PCI commands requires the FPGA to be configured, so meeting this spec requires an extremely fast configuration solution be used. This is achieved by using a CPLD that reads a stored bitstream out of flash and configures the FPGA over a 32-bit SelectMAP interface clocked at 100MHz.

Digilent designed the firmware for the CPLD so that four different bitstreams can be stored in the flash. On power-up (if JP1 is not loaded), one of these bitstreams is read by the CPLD and used to program the FPGA. The bitstream that is used is determined by a non-volatile register called the “Boot Section Select” (BSS) register. If the BSS register points to a section that does not contain a valid bitstream, the Error LED (LD5) is illuminated and the FPGA is not configured.

After the FPGA has been programmed (whether by JTAG or from parallel flash), it is possible for the design to trigger the CPLD to reconfigure the FPGA with one of the stored bitstreams. This is done by selecting the desired bitstream with the CPLD_IMGSEL0 and CPLD_IMGSEL1 pins and then driving CPLD_RECONFIG high. For example, if CPLD_IMGSEL0='0' and CPLD_IMGSEL1='1' when CPLD_RECONFIG is driven high, the FPGA will immediately be reprogrammed with the bitstream stored in boot section 2. If the boot section indicated by the CPLD_IMGSEL pins does not contain a valid bitstream, then the bitstream in the section pointed to by the BSS register is used and the Fallback LED (LD6) is illuminated. Note that the CPLD_IMGSEL2 pin is not used.

After the CPLD has successfully programmed the FPGA, the boot section that the bitstream was loaded from is indicated using the IMGCUR0 (LD7) and IMGCUR1 (LD8) LEDs. The IMGCUR2 LED (LD9) is not used.

Bitstreams are programmed into or read from flash using the dsumecfg tool included with the Adept Utilities package. You can also use this tool to set the BSS register. The Adept Utilities toolset can be downloaded for free from the Adept 2 page on the Digilent Website. Full documentation on the dsumecfg tool is included with the installation. The following should be kept in mind when using the dsumecfg tool:

  • In order to use dsumecfg, you must have the NetFPGA-SUME connected to your computer via the USB-JTAG port.
  • To decrease programming times, we highly recommend enabling bitstream compression in Vivado or ISE when you generate your bitstream.
  • dsumecfg will not work properly if an FMC card that inserts a JTAG device into the scan chain is attached to the NetFPGA-SUME.

Memory

DDR3 SODIMM

The NetFPGA-SUME board comes with two Micron MT8KTF51264HZ-1G9 4GB DDR3 SDRAM SODIMM, which employs an 932.84MHz 64bit-wide data bus capable of operating at a data rate of 1866MT/s. Project development with the SDRAM involves using the Xilinx Memory Interface Generator (MIG) in Vivado Design Suite. The interface is automatically configured by the MIG for use with the AXI4 system bus and provide a fixed 4:1 memory to bus clock ratio. The input clock for both SDRAM SODIMMs is a 233MHz clock generated by Discera DSC1103 Low Jitter Precision LVDS Oscillator. The clock period of SDRAM is configured to 1177ps (849.62MHz), equivalent to 1700MT/s, due to the read margin issues. Please refer to Xilinx Answer Record AR61853 for further information. The NetFPGA-SUME uses a VCCAUX-IO of 2.0V to support high performance DDR3 frequency settings. Please see Xilinx 7-Series FPGAs Memory Interface Solutions User Guide (UG586) and the micron 1GB, 2GB, 4GB (x64, SR) 204-Pin DDR3L SODIMM data sheet for more details. The DDR3 project in unit test project in netfpga repository provides a good starting point for project development.

QDRII+ SRAM

Three Cypress CY7C25652KV18 Quad Data Rate II+ (QDRII+) SRAMs are provided for applications that require high speed, low latency memory. Each component provides a 36 bit wide data bus and has a density of 72 Megabits. Common applications include FIFO buffers and look-up tables. The notion of “Quad” data rate comes from the ability to simultaneously read from a unidirectional read port and write to a unidirectional write port on both clock edges. The QDRII+ SRAMs on NetFPGA-SUME board are capable of operating at up to 500MHz to yield data transfer rates of up to 1GT/s per 36-bit wide data bus. The Xilinx Memory Interface Generator (MIG) is able to generate and configure an native interface into the QDRII+ via the user friendly wizard tool. More information regarding the QDRII+ memory part and the Xilinx MIG tool can be found in the Cypress CY7C25632KV18/CY7C25652KV18 data sheet, the Cypress Application Note QDR-II, QDR-II+, DDR-II, DDR-II+ Design Guide (AN4065), and the Xilinx 7-Series FPGAs Memory Interface Solutions User Guide (UG586).

QDRA and QDRB share FPGA bank 17, which means that in order to access them simultaneously, a bank sharing solution must be used that extends beyond the default functionality of the MIG. This solution is still currently being developed. Please refer to Xilinx Answer Record 41706 for further information.

Storage

FLASH

Onboard parallel flash is available for storing FPGA bitstreams. For information on writing bitstreams to flash and configuring the FPGA from stored bitstreams, see the section titled “Configuration using Parallel Flash”.

MicroSD Card

The microSD card connector on NetFPGA-SUME board provides a removable non-volatile storage resource. This connector supports a microSD memory card and meets all physical layer requirements of both SPI and SD bus protocols. It supports the UHS-I pin assignment standard (but not UHS-II) and provides high speed signaling at 3.3V to support SC, HC, and XC class SD cards. Please see SD Specifications Part 1 Physical Layer Simplified Specification by the Technical Committee of the SD Card Association for more details regarding the use of SD memory cards with this connector.

SATA

The NetFPGA-SUME board provides two SATA ports which are SATA-III compatible (6Gbps). Two GTX transceivers (Lane 0,1 on Bank 116) are dedicated to these two ports with a master clock of 150MHz generated by Discera DSC1103 Low Jitter Precision LVDS Oscillator. SATA PHY controller can be generated using Xilinx GTX Transceiver Wizard. Please refer to Xilinx Answer Record AR 53364, AR 44587 and UG769 7-Series FPGAs Transceivers Wizard v2.6 User Guide for more information.

PCI Express

The NetFPGA-SUME is designed with a PCI-Express form factor to support interconnection with common processor motherboards. Eight of the FPGA's high speed serial GTX transceivers are dedicated to implementing eight-lanes of Gen. 3.0 (8 GB/s) PCIe communications with a host processing system (there is no support for Gen3 x4 configuration). These transceivers work in conjunction with the on-chip 7 Series Integrated PCI Express Block and synthesizable on-chip logic to provide a scalable, high performance PCI Express I/O core. Please refer to the Xilinx 7-Series FPGAs Integrated Block for PCI Express V2.0 (PG054) product guide and 7-Series FPGAs GTX/GTH Transceivers (UG476) user guide for more information.

SFP+ 10Gbps Ethernet Interface

The NetFPGA-SUME board provides four enhanced small form factor pluggable (SFP+) connectors, each supports 10Gbps. Four of the FPGA’s high speed serial GTX transceivers on Bank 119 are dedicated to four SFP+ ports. These connectors are capable of implementing 10GBase-SR/LR Ethernet Protocols.

I2C

The architecture of Inter-Integrated Circuit (I2C) buses on NetFPGA-SUME board is illustrated in the figure below.

Virtex-7 FPGA is the master on the I2C bus, and connected to an I2C multiplexor (PCA9548) via a level shifter (FXMA2102). THe PCA9548 has 8 downstream ports which are wired to SFP+ transceivers, DDR3 SODIMMs, FMC connectors, any-frequency precision clock generator (SI5324) and the PMBus (a subset of I2C protocol) interface of power management chips (LTC2974). In the figure, the hexadecimal numbers on the double arrow is the control register value to select the corresponding downstream port, and the red hex number besides each device is the I2C slave address of each device.

Clocking

On-board oscillators and clock generators support various board subsystems. Transceivers connected to SFP+ interfaces (Bank 119) and QTH connectors (Bank 117, 118) share the same SFP_CLK (Bank 118 Clk0) generated by an any-frequency precision clock multiplier/jitter attenuator (IC20 SI5324). Under most cases when SFP+ interfaces are configured to be 10Gb Ethernet ports, the output clock frequency of SI5324 is set to be 156.25MHz (Please refer to reference design in NetFPGA-SUME repository for recommended configuration values). The transceivers for both SATA connectors (Lane 0, 1 Bank 116) run from a 150MHz SATA_CLK (Bank 116 CLK0) generated by a low-jitter LVDS Oscillator (IC21 DSC1103). FPGA_SYSCLK (system clock for FPGA), QDRII_SYSCLK (shared by QDRII+ memory A and B), QDRIIC_SYSCLK (used by QDRII+ memory C) are 200MHz clocks generated by another low-jitter LVDS Oscillator (IC16 DSC1103) level-shifted by a low-jitter low-skew clock buffer (IC17 SI5330). DDR3_SYSCLK (shared by two DDR3 SODIMMs) is a 233.33MHz clock generated by the third low-jitter LVDS oscillator (IC18 DSC1103). The PCIE-CLK needs external 100MHz clock used specifically for PCI-Express Gen3 x8 design and FMC_GBT_CLK0/1, FMC_CLK0/1 connected to FMC connectors reserved for user-defined FMC peripheral boards.

The following table is a summary for all the clock resources available on board:

Clock Name Source Destination Frequency Common Useage
FPGA_SYSCLK IC16 DSC1103, IC17 SI5330 FPGA (H19, G18) 200MHz General Purpose
QDRII_SYSCLK IC16 DSC1103, IC17 SI5330 FPGA (AD32, AD33) 200MHz Used by MIG for QDRIIA, QDRIIB
QDRIIC_SYSCLK IC16 DSC1103, IC17 SI5330 FPGA (AU14, AU13) 200MHz Used by MIG for QDRIIC
SATA_SYSCLK IC21 DSC1103 FPGA (MGTREFCLK0_116:T8, T7) 150MHz Used by SATA transceivers (Bank 116 Lane 0/1)
SFP_CLK IC20 SI5324 FPGA (MGTREFCLK0_118: E10, E9) 156.25MHz (Configurable) Shared by transceivers of SFP+ and QTH
DDR3_SYSCLK IC18 DSC1103 FPGA (E34, E35) 233.33MHz Shared by MIG for DDR3A, DDR3B
PCIE-CLK PCI-E Connector FPGA (MGTREFCLK1_115: AB8, AB7) 100MHz Used by PCI-Express Core
FMC_GBT_CLK0 FMC Connector FPGA (MGTREFCLK0_112: AT8, AT7) - Used by FMC transceivers (Bank 111, 112, 113)
FMC_GBT_CLK1 FMC Connector FPGA (MGTREFCLK1_112: AU10, AU9) - Used by FMC transceivers (Bank 111, 112, 113)
FMC_CLK0 FMC Connector FPGA (AR27, AT27) - Used by FMC card
FMC_CLK1 FMC Connector FPGA (AV34, AV35) - Used by FMC card

Expansion Interfaces

FMC

The NetFPGA-SUME board includes a VITA-57 compatible FMC (FPGA Mezzanine Card) carrier connector. A High Pin Count (HPC) connector is used to provide the maximum possible compatibility with a variety of commercially available mezzanine cards. Select I/O ports on the XC7V690T are connected to all of the standard Low Pin Count (LPC) signals on the connector, due to the limitations of the FFG1761 package. All the I/O ports connected to FMC connector only supports 1.8V logic. All 10 differential send/receive pairs for GTX transceivers are also supported.

Please refer to the American National Standards Institute ANSI/VITA 57.1 FPGA Mezzanine Card (FMC) Standard for additional details regarding standard FMC module and carrier requirements. Refer to Appendix B of this document for specific I/O constraints relating FPGA pins to their associated FMC control and connector pins.

QTH

The NetFPGA-SUME board includes a Samtec 0.5 mm-pitch QSH/QTH family of high-speed board-to-board communication. The QTH connector wires out 8 transceivers from Virtex-7 FPGAs and tested at a speed of 12.5Gbps. Please refer to Samtec website for additional details regarding the QTH connector's specification, footprint, etc.

Pmod

Pmod Port

The NetFPGA-SUME board also provides a Pmod port for peripheral extension. The Pmod port is arranged as a 2×6 vertical, 100-mil female connector that mates with standard 2×6 pin headers. Each 12-pin Pmod port provides two 3.3V VCC signals (pins 6 and 12), two Ground signals (pins 5 and 11), and eight logic signals, as shown above. The VCC and Ground pins can deliver up to 1A of current. Pmod data signals are not matched pairs, and they are routed using best-available tracks without impedance control or delay matching.

The signals on the Pmod port are connected to the FPGA via two 4-bit dual-supply bus transceivers (IC1 and IC2, part number SN74AVC4T774) with configurable voltage translation and 3-state outputs. You need to specifically set DIR for each pin to control the signal direction. The bus transceiver is enabled by driving OE pin of the bus transceiver low.

Basic I/O

The NetFPGA-SUME board includes two LEDs (LD0 and LD1) and two pushbuttons (BTN0 and BTN1) that can be used for basic user I/O. These can be useful for debugging designs. The pushbuttons are connected to the FPGA via series resistors to prevent damage from inadvertent short circuits (a short circuit could occur if an FPGA pin assigned to a pushbutton or slide switch was inadvertently defined as an output). The two pushbuttons are “momentary” switches that normally generate a low output when they are at rest, and a high output only when they are pressed.

The two LEDs are green and are illuminated when driven high. It is possible to control the brightness by providing a pulse width modulated signal that varies the duty cycle from 0% to 100%.

An additional red pushbutton (BTN3, labeled PROG) is attached to the PROGRAM_B pin of the Virtex-7 FPGA. Pressing this button will clear the configuration inside the FPGA and cause the DONE pin to go low. If the mode jumper is not shorted, this will also trigger the CPLD to reprogram the FPGA with a bitstream stored in flash (See the “FPGA Configuration” section for more info).

LD4 is attached to the DONE pin of the FPGA, and is illuminated whenever the FPGA is configured with a valid bitstream.