PSI Home

GFA Home

ABK Home

SLS Home

SLS Controls Home

	Diagnostic

	SLS Diagnostics • DBPM-BO • DBPM • FOFB • MBF

	Proton Diagnostics

	European X-FEL

	SwissFEL

	Intranet

	Publications

	Tools Inventory Database

	KTRANS

	DI ELOG

Author: Thomas Schilcher
Phone: +41 56 310 4593
Updated: 09.09.2015

Printer friendly version

WS9003 Fiberlink FPGA User's Manual

FPGA Firmware Version: 030810-1
Author: boris.keil@psi.ch
Revision: 1.2
Date: 10/08/2003

1. Introduction
2. Control Registers
2.1 Configuration and Reset Registers
2.2 Trigger/Interrupt Output Enable Registers
2.3 Interrupt Mask Registers

3. Status Registers
4. Example Application

1. Introduction

The WS9003 fiberlink FPGA can be accessed via DSP data/address bus. Only the data lines D[19...16] of the bus are connected to the FPGA, therefore all registers have a width of only 4 bit. In the tables below D16 is named bit 0, D17 is bit 1 etc. The FPGA provides 8 control registers and 6 status registers. Control registers provide write and read access, status registers are read-only. The data bus bits D[47...20] for read access of control and status registers are undefined and should be masked out. Signal names that begin with a slash "/" are active low signals, all other signals are active high. According to the signal name convention of the original FPGA design from Wiese GmbH the prefixes "TX_" ("RX_") denote transmission (reception) of link port data to (from) the Sharc DSP, while the prefixes "TXFOL_" ("RXFOL_") denote transmission (reception) of data to (from) the fiber optics link.

The FPGA firmware assumes that the DSP has an external 40 MHz clock and that the FPGA has an external 80 MHz clock. The DSP must be programmed for 4 WAIT states plus one hold/idle cycle for any data bus read or write access of FPGA registers. The link port from DSP to FPGA should use 40 MHz link port clock rate, i.e. a transfer rate of 20 MByte/s. 80 MHz clock rate will only work if the signals on the PCB are sufficiently "clean", which requires good PCB layout with sufficiently short signal traces and/or terminators.

2. Control Registers

After an external reset of the FPGA all control registers are set to 0, which disables the fiberlink (including the baud rate clock). FIFOs in the transmission and reception section of the FPGA are also reset by an external reset signal. Futhermore, the FIFOs and the state machine of the serialiser/deserialiser ICs can be reset individually during normal operation of the FPGA by control register bits.

2.1 Configuration and Reset Control Registers


Control Register No.	Address Offset	Bit No.	Signal Name	Description
0	0x0	0	/EN_TX_LINK1	0: use TX link port 1 1: use TX link port 2
		1	/EN_RX_LINK3	0: use RX link port 3 1: use RX link port 4
		2	SEL_LOW_CLK	0: fiberlink 400 MBaud (320 Mbit/s max. net data rate) 1: fiberlink 200 MBaud (160 MBit/s max. net data rate) The fiberlink baud rate should be selected once after power-up, before the fiberlink transmission clock (bit 3 of this register) is enabled and while all three reset bits in control register 1 are still active (low). The baud rate should not be changed any more after the transmission clock was enabled.
		3	TXFOL_STRBEN_LOC	0/1: Disable/Enable the fiberlink transmission clock. Should be enabled after power up and not be disabled ever again.
1	0x1	0	/TX_RESET_LOC	0: disable and reset data transfer (including a FIFO reset) from fiberlink to DSP link port. 1: enable data transfer ... The internal reset signal is activated when the /TX_RESET_LOC (active low) bit is set from 1 to 0, but it is deactivated 400 ns after the bit is set from 0 to 1. This guarantees that the reset pulse is long enough to obtain a defined FPGA state. The status register bit TX_RESET (active HIGH) may be used to monitor when the internal reset signal has becomes inactive, i.e. when the 400 ns delay has expired. Please note that /TX_RESET_LOC does not reset the error flags for the data transfer from fiberlink to DSP. They have to be reset separately via TX_RESET_ERR.
		1	/RX_RESET_LOC	0: disable and reset data transfer (including a FIFO reset) from DSP link port to fiberlink. 1: enable data transfer ... The internal reset signal that is generated by this bit remains low for additional 400 ns after /RX_RESET_LOC (active low) changed from 0 to 1 (similar to /TX_RESET_LOC). The status register bit RX_RESET (active HIGH) may be used to monitor when the internal reset signal has becomes inactive, i.e. when the 400 ns delay has expired. Please note that /RX_RESET_LOC does not reset the error flags for the data transfer from DSP to fiberlink. They have to be reset separately via RX_RESET_ERR.
		2	/SERDES_RESET_LOC	0: reset the state machine in the HP serialiser/deserialiser chip set. 1: allow the state machine to start data transmission (after a startup sequence of the state machine for PLL locking etc.). The internal reset signal that is generated by this bit remains low for additional 400 ns after /SERDES_RESET_LOC (active low) changed from 0 to 1. The status register bit SERDES_RESET (active HIGH) may be used to monitor when the internal reset signal has becomes inactive, i.e. when the 400 ns delay has expired. After a reset of the fiberlink transmitter (SERialiser) and receiver (DESerialiser) ICs it may take many milliseconds until the links are ready for data transfer again. This can be checked by polling the status register bits RXFOL_LINKRDY (deserialiser) and TXFOL_RFD_STAT (serialiser).
		3	-	unused
2	0x2	0	TX_RESET_ERR	Reset the error flag bits for the data transfer from fiberlink to DSP. The following error flags will be reset to 0: RXFOL_ERR_MEM TX_FULL_ERR_MEM TX_16BIT_MISS_ERR RXFOL_LINKRDY_ERR
		1	RX_RESET_ERR	Reset the error flag bits for the data transfer from DSP to fiberlink. The following error flags will be reset to 0: TXFOL_RFD_ERR_MEM RX_FULL_ERR_MEM
		2	RESET_REM_ALM_FULL	1: Reset the flag which indicates that the FIFO of the FPGA at the other end of the transmitting fiberlink is almost full. This reset bit is only required for testing purposes and should normally always be 0. By setting it to 1 (continously) the FIFO-full handshaking between transmitter and FPGA at the other end of the fiberlink is disabled, which may lead to an overflow of the FIFO from fiberlink to DSP for the FPGA at the other end of the fiberlink.
		3	-	unused

2.2 Trigger/Interrupt Output Enable Control Registers


Control Register No.	Address Offset	Bit No.	Signal Name	Description
3	0x3	0	DSP_IRQ1_EN	0/1: DSP interrupt signal IRQ1 disabled/enabled. Disabled means that the signal is High-Z so that the IRQ line may be driven by another signal source. In High-Z state the signal level may be undefined if there is no pullup resistor (which is not documented by the manufacturer of the DSP board). The DSP should disable the interrupt as long as the interrupt line is not driven.
		1	DSP_IRQ2_EN	0/1: DSP interrupt signal IRQ2 disabled/enabled ...
		2	TRIG_A_OE	0: Trigger output A High-Z 1: Trigger output A driven by FPGA. !!! WARNING !!! If the trigger line is driven by more than one source (e.g. by two FPGAs or by one FPGA and the SLS timing module on the DSP J2 connector) the FPGA may be damaged.
		3	TRIG_B_OE	0: Trigger output B High-Z 1: Trigger output B driven by FPGA. !!! WARNING !!! If the trigger line is driven by more than one source (e.g. by two FPGAs or by one FPGA and the SLS timing module on the DSP J2 connector) the FPGA may be damaged.
4	0x4	0	TRIG_AIN_INV	Invert trigger input A (bidirectional pin, may be used as trigger input input or output). If this bit is 0 (1) the input signal is active high (low).
		1	TRIG_BIN_INV	Invert trigger input B (bidirectional pin, may be used as trigger input input or output). If this bit is 0 (1) the input signal is active high (low).
		2	TRIG_AOUT_INV	Invert trigger output A (bidirectional pin, may be used as trigger input input or output). If this bit is 0 (1) the output signal is active high (low). As long as the trigger source signal is constant you can toggle this bit in order to generate a trigger pulse via software.
		3	TRIG_BOUT_INV	Invert trigger output B (bidirectional pin, may be used as trigger input input or output). If this bit is 0 (1) the output signal is active high (low). As long as the trigger source signal is constant you can toggle this bit in order to generate a trigger pulse via software.

2.3 Interrupt Mask Control Registers

The following control registers are used to determine which of the bits in the status registers 0 and 1 can generate a DSP interrupt. The DSP interrupt lines /IRQ1 and /IRQ2 are active (low) when a logic-AND of the status registers 0 (1) and of the control register 5 (6) is not zero, i.e. when a status bit is set and the respective interrupt mask bit is also set. In addition, the trigger I/Os A and B can also generate DSP interrupts if they are used as inputs, i.e. if the trigger lines have an external driver (e.g. an SLS timing backplane module). If the trigger lines A and B are not driven by an external signal source they may be used as outputs (by setting the control register bits TRIG_A_OE or TRIG_B_OE to 1), and the /IRQ1 signal (active low) is also sent to these outputs. The trigger output A (B) is active high if the control register bit TRIG_AOUT_INV (TRIG_BOUT_INV) is 0, otherwise it is active low (like the /IRQ1 output signal).


Control Register No.	Address Offset	Bit No.	Signal Name	Description
5	0x5	0	IRQ1_USE_TRIG_A_IN	0/1: don't/do use trigger input A to generate a DSP IRQ1 interrupt. Take care that the polarity of the trigger input (may be changed via TRIG_AIN_INV) and the DSP interrupt mode (level or edge sensitive) have suitable values.
		1	IRQ1_USE_TRIG_B_IN	0/1: don't/do use trigger input B to generate a DSP IRQ1 interrupt ...
		2	IRQ2_USE_TRIG_A_IN	0/1: don't/do use trigger input A to generate a DSP IRQ2 interrupt ...
		3	IRQ2_USE_TRIG_B_IN	0/1: don't/do use trigger input B to generate a DSP IRQ2 interrupt ...
6	0x6	0	IRQ1_USE_RXFOL_ERR_MEM	0/1: don't/do use status bit RXFOL_ERR_MEM to generate a DSP IRQ1 interrupt.
		1	IRQ1_USE_TX_FULL_ERR_MEM	0/1: don't/do use status bit TX_FULL_ERR_MEM (active high) to generate a DSP IRQ1 interrupt.
		2	IRQ1_USE_TX_16BIT_MISS_ERR	0/1: don't/do use status bit TX_16BIT_MISS_ERR to generate a DSP IRQ1 interrupt.
		3	IRQ1_USE_RXFOL_LINKRDY_ERR	0/1: don't/do use status bit RXFOL_LINKRDY_ERR (active high) to generate a DSP IRQ1 interrupt.
7	0x7	0	IRQ1_USE_TXFOL_RFD_ERR_MEM	0/1: don't/do use status bit TXFOL_RFD_ERR_MEM (active high) to generate a DSP IRQ1 interrupt.
		1	IRQ1_USE_RX_FULL_ERR_MEM	0/1: don't/do use status bit RX_FULL_ERR_MEM (active high) to generate a DSP IRQ1 interrupt.
		2	-	unused
		3	-	unused

3. Status Registers

The first six bits in the following registers are "sticky" bits that are set to 1 (and remain 1) by an error condition, even if this error condition occurs only for a very short period of time. These bits may generate DSP interrupts. For convenience, they have the same order than the interrupt mask register bits (control registers 5 and 6) as described above. The sticky status register bits have to be reset to 0 manually by toggling the control register bit TX_RESET_ERR or RX_RESET_ERR (see below). If the sticky bits generate interrupts, the interrupt handler should read the status register and toggle TX_RESET_ERR and/or RX_RESET_ERR immediately afterwards in order to clear the interrupt condition. The delay between reading and clearing the status bits should be short enought to avoid that a sticky bit changes from 0 to 1 in between, which would result in an undetected additional interrupt condition.

Except for sticky bits that may generate interrupts the status registers below also contain non-sticky status bits that simply reflect the actual state of a hardware signal at the point in time when the DSP reads the register. Readout of these bits makes only sense when the signal levels are known to be constant on a time scale that is much longer than a DSP read cycle, while short pulses of the signals (in the order of one or a few 80 MHz clock periods) can only be detected safely by the above "sticky" bits.


Status Register No.	Address Offset	Bit No.	Signal Name	Description
0	0x20	0	RXFOL_ERR_MEM	RX fiber optics link error: the bit is set to 1 (and remains 1) if an invalid data frame (that is neither a 20-bit data frame nor a control or fill frame) was received via fiberlink. This bit indicates transmission faults of the fiber optics transceivers. It can be reset to 0 via the control register bit TX_RESET_ERR
		1	TX_FULL_ERR_MEM	This bit is set to 1 (and remains 1) when the write access to the FIFO for data transmission from fiberlink to DSP fails because the FIFO is already full. This should never happen if the handshaking between the two FPGAs at each end of the fiberlink works properly. The bit may be reset to 0 via TX_RESET_ERR.
		2	TX_16BIT_MISS_ERR	The WS9003 FPGA transmits 16-bit data words in pairs, with no control or fill word in between (since it receives 32-bit data words via link port from the DSP). This bit is set to 1 (and remains 1) when the fiberlink did not receive an even number of subsequent data words (framed by words that are not data words), which may only happen in case of transmission errors on the fiberlink. The last data word of such an odd series of words is discarded. This guarantees that there is no continous 16-bit shift in the data stream from FPGA to sharc, which would lead to continous data errors. The TX_16BIT_MISS_ERR bit may be reset to 0 via TX_RESET_ERR.
		3	RXFOL_LINKRDY_ERR	This bit is set to 1 (and remains 1) when the fiberlink receiver is not ready to receive data. This may happen if the data link is broken, e.g. for a short time after power-up or when a fiber optics cable is disconnected. The bit may be reset to 0 via TX_RESET_ERR.
1	0x21	0	TXFOL_RFD_ERR_MEM	This bit is set to 1 (and remains 1) when the fiberlink transmitter (i.e. the HP serialiser) is not ready to send data. The signal should changes nearly simultaneously with RXFOL_LINKRDY_ERR, since both transmission and reception of the fiberlink are handled by a single state machine that is located in the HP deserialiser. The bit may be reset to 0 via the control register bit RX_RESET_ERR.
		1	RX_FULL_ERR_MEM	This bit is set to 1 (and remains 1) when the FIFO for data transfer from DSP to fiberlink is full. This should never happen.The bit may be reset to 0 via the control register bit RX_RESET_ERR.
		2	-	unused
		3	-	unused
2	0x22	0	RXFOL_LINKRDY	0 / 1: fiberlink receiver (deserialiser) not ready / ready for data reception. This bit reflects the actual state of the signal from the deserialiser, in contrast to RXFOL_LINKRDY_ERR which changes to 1 and remains 1 when the receiver is NOT ready until it is reset via TX_RESET_ERR.
		1	RXFOL_ERR	This bit reflects the actual state of the RX fiber optics link error signal from the HP deserialiser at the point in time when this bit is read via DSP, in contrast to RXFOL_ERR_MEM which is set to 1 by the deserialiser error signal and has to be reset manually.
		2	TXFOL_RFD_STAT	0 / 1: fiberlink transmitter (serialiser) not ready / ready for data transmission. This bit reflects the actual state of the signal from the serialiser, in contrast to TXFOL_RFD_ERR_MEM which changes to 1 and remains 1 when the transmitter is NOT ready until it is reset via RX_RESET_ERR.
		3	REM_FIFO_ALM_FULL	This bit indicates that the FIFO of the FPGA at the other end of the fiberlink that is used for data tranfer from fiberlink to DSP is almost full (i.e. more than half full). If the bit is 1 the data transfer to the other FPGA is stopped (the LACK signal to the DSP is set to 0) until REM_FIFO_ALM_FULL goes to 0 again. The bit is transferred between the two FPGAs of a fiberlink typically every 8 transmission strobes by bit 0 of a control word, while normal link port data is transferred via data words. Data and control words are distinguished by the 20-bit deconding/encoding scheme of the serialisers/deserialisers. This bit is not sticky, it reflects the actual state of the remote FIFO.
3	0x23	0	RX_ONEWORD	These two bits should be 0 after the DSP has completed the transfer of a data package to the FPGA (via link port) and the FPGA has transmitted all data via fiberlink. RX_ONEWORD is 1 when there is exactly 1 16-bit word in the 16-bit FIFO for data transmission from DSP to fiberlink. RX_ONEWORD should be read while no data words are written to or read from the FIFO. RX_BADNIBBLE is 1 if the number of nibbles transferred from DSP to FPGA was not a multiple of 4. It should be read while no data is being transferred from DSP to FPGA via link port. Since the DSP transfers 32-bit words (in eight 4-bit nibbles) via link port to the FPGA, the FPGA transmits 16-bit data words only in pairs (directly after each other) to detect 32-bit alignment errors at the receiving end of the fiberlink. As long as the link port from DSP to FPGA works properly the FIFO should be empty and RX_ONEWORD should be 0 after the transmission of a set of data words from DSP to fiberlink has finished (e.g. a DMA transfer) and after all FIFO words have been transmitted. If RX_ONEWORD and/or RX_BADNIBBLE remain 1 this means that the link port does not work properly and that the number of nibbles transferred from DSP to FPGA was not a multiple of 8. This may be caused by hardware faults or bad electronics design ("ringing" on link port clock lines, ...).
		1	RX_BADNIBBLE
		2	-	unused
		3	-	unused
4	0x24	0	TX_EMPTY	1: 32-bit FIFO for link port data transmission from fiberlink to DSP is empty
		1	TX_HALF	1: 32-bit FIFO for link port data transmission from fiberlink to DSP contains 32 or more words (FIFO size: 64 words)
		2	RX_EMPTY	1: 16-bit FIFO for link port data transmission from DSP to fiberlink is empty
		3	RX_FULL	1: 16-bit FIFO for link port data transmission from DSP to fiberlink is full (size: 32 16-bit words)
5	0x25	0	TX_RESET	These three bits are 1 when the respective internal reset signal that is generated by /TX_RESET_LOC (bit 0), /RX_RESET_LOC (bit 1) or /SERDES_RESET_LOC (bit 2) is active, otherwise they are 0. Since an internal reset signal remains active for additional 400 ns after the respective control register bit that generates the reset was set from 0 (= reset enable) to 1 (= reset disabled), these three status bits may be used to monitor when the 400 ns delay for each of the three reset signals has expired (instead of using a calibrated software delay loop).
		1	RX_RESET
		2	SERDES_RESET
		3	-	unused

4. Example Application

This section gives an example of how to use the above FPGA registers.

After power-up with an external reset (that sets all control register bits to 0) the DSP may perform the following initialisation sequence:


Step	Read/ Write	Address Offset	Value	Comment
1	-	-	-	Disable & reset both sharc link ports (RX, TX) in the DSP, disable IRQ1 and IRQ2 in the DSP. Set all WS9003 FPGA control registers to 0 if the FPGA was not reset externally.
2	W	0x0	0x8	Enable the serializer strobe (bit 3 = 1) that determines the baud rate. Use 400 MBaud (bit 2 = 0). Use TX_LINK1 and RX_LINK3 (bit 0 = 0, bit 1 = 0).
3	W	0x1	0x7	Disable reset signals for serializser/deserialiser (bit 2 = 1), fiberlink transmitter (bit 1 = 1) and fiberlink receiver (bit 0 = 1) section.
4	R (poll loop)	0x25	-	Wait until bits 0,1 and 2 of status register 5 are all 0, which indicates that all three internal reset signals (that remain active for 400 ns after the respective control register reset bits have changed to 1) are inactive. The 400 ns delay is generated internally to guarantee a defined state of the FPGA no matter how fast the user toggles the control register bits that generate the internal reset signals.
5	W	0x2	0x3	Toggle the reset signals for the "sticky" status register error bits.
6	W	0x2	0x0
7	R	0x20	-	If bit 3 of status reg. 0 is 1 (fiberlink receiver not ready) or bit 0 is 1 (fiberlink receiver errors): go back to step 5.
8	R	0x21	-	If bit 0 of status reg. 1 is 1 (fiberlink transmitter not ready): go back to step 5.
9	W	0x4	...	Invert trigger inputs and/or outputs if required
10	W	0x5, 0x6, 0x7	...	Set interrupt mask registers as required
11	W	0x3	...	Enable interrupt and trigger outputs as required (are high-Z after power-up). WARNING: the trigger outputs must not be driven by more than 1 signal source. Do not enable the outputs if the signal already has a driver (e.g. on both FPGAs), or the FPGA (and the other driver) may be damaged.
12		-	-	Connect the IRQ1 and IRQ2 interrupts of the DSP to appropriate interrupt service routines that read and clear the respective FPGA interrupt status registers. It is recommended to use level-sensitive interrupts to detect error conditons via "sticky" status register bits. Interrupts that use only the trigger inputs A and/or B may use either edge-sensitive interrupts (e.g. for very long trigger pulses) or level- sensitive interrupts (e.g. for sufficiently short trigger pulses that are guaranteed to become inactive before the interrupt routine is finished).
13		-	-	Now the fiber link is working, and the link ports may be enabled and used for data transfer.

Author: Thomas Schilcher Phone: +41 56 310 4593 Updated: 09.09.2015