Project topics and materials
Get Complete Project Material File(s) Now! »
AMBA bus, GRLIB and LEON3
One of the main contributions from this thesis is the implementation of the rmware used for the SpaceFibre IOD board developed in Thales Alenia Space. The rmware, implemented in a COTS Static Random Access Memory (SRAM) FPGA is based on the LEON3 processor used as a soft-core to execute the control software, along with several peripheral controllers from the GRLIB IP library that allow the board to perform its desired functionality. The SoC uses the Advanced Microcontroller Bus Architecture (AMBA) 2.0 bus to connect to all components of the System on Chip.
AMBA bus
The AMBA bus is an open standard developed by ARM Ltd. for the interconnection of on-chip units in a SoC. It is currently the de-facto standard interface used for the devel-opment of hardware IPs in the space sector. There are several interfaces in the AMBA speci cation, where the Advanced Extensible Interface (AXI), AMBA High-performance Bus (AHB) and Advanced Peripheral Bus (APB) are the most widely used. This thesis uses the two latter for the interconnection of the SoC elements. Both buses are connected to a controller and arbiter that maps the control and status registers of the peripherals to the processor’s memory space, with the memory controller also connected to the AHB bus. Both buses are technology-independent and designed for portability. Since they are sometimes implemented in FPGAs, where high impedance internal bu ers are not avail-able, they are implemented in the form of a multiplexed bus: each master drives the bus signals, and the output of the current bus master is selected by the multiplexers and sent to the slaves input. Similarly, the output of the active slave is selected and sent to the masters as shown in gure 2.1.
The AHB bus is a high-performance multi-master bus that allows for burst reads and writes. It is used for peripherals that need to frequently transmit and receive larger amounts of data, such as the memory controller |managing the transactions between the processor and the SRAM and ash PROM memories|. The bus is multi-master and multi-slave, controlled by a global arbiter. The arbiter receives the address information needed to communicate with each master and, by means of a grant signal, allows one master at a time to have control over the bus. Similarly, the slaves provide addressing information, and have one dedicated signal to access them while they are selected.
The APB bus, on the other hand, is controlled through an AHB-to-APB bridge, which is an AHB slave and the only APB master in the bus. It is a low-cost bus|in terms of logic| without arbiter used for low-bandwidth and simpler peripherals, such as timers, serial ports and interrupt controllers.
LEON3
The LEON3 processor [6] was designed and developed by Cobham Gaisler A.B., a Swedish company founded by Jiri Gaisler, the creator of the Leon architecture. It is based on the SPARC V8, a Reduced Instruction Set Computer (RISC) architecture, and o ered in GRLIB under a GNU General Public License (GPL) as synthesizable VHDL. It uses an AMBA AHB 2.0 interface, which is focused towards the development of SoC designs. Be-sides, it o ers advanced features such as a 7-stage pipeline, con gurable caches, debugging support, high performance |1.4 Dhrystone Million Instructions per Second (DMIPS) for each MHz| and it is highly con gurable through means of VHDL generics.
GRLIB
One of the key factors for the success of the LEON3 adoption in the space industry is that, along with it, Cobham Gaisler provides an extensive open-source IP library. All IPs come with an AMBA 2.0 compliant interface, and can communicate with a LEON3-based SoC seamlessly thanks to the included AHB and APB controllers. With the exception of the SpaceFibre integrated in this work, the rest of the components used to form the SoC that composes the rmware for the ight board are directly integrated from GRLIB and freely available.
In order to integrate a GRLIB IP into a design, it is enough to include the necessary library into the compilation of the Hardware Description Language (HDL), and to declare the IP as a component and connect the AMBA signals at the top level of the design. Customization is completely managed by generics. The IPs are also easily managed from the software running on the LEON3, since their status and control registers are memory-mapped in the processor’s address space. Instructions to install and use the library, as well as documentation on how the AMBA bus works are detailed in [1]. Every IP available under GPL license is documented in [7].
Additional tools
Besides GRLIB, Cobham Gaisler provides several tools to facilitate the development of SoCs. The one that was used the most is GRMON2 [8], a debugging tool that communi-cates the debug-support unit of the processor, providing useful services such as obtaining system plug-and-play information, displaying memory contents for the whole address-space and writing to memory in the whole address-space. This allows manual set-up of peripherals and preliminary testing without the need for any kind of software. Likewise, support for Common Flash Memory Interface (CFI) commands is included, facilitating to erase and program the ash rom used to boot the processor. Similarly, it also includes the capability to load software code to the RAM memory and execute it along with the GDB tool for debugging.
Finally, a cross-compiler to generate LEON3 binaries as bare-metal software |i.e., that runs without an operating system| (Bare-C Cross-Compiler) [9] and boot images(MKPROM2) [10] facilitated the work carried out in this thesis.
Radiation upsets on SRAM-based FPGAs
The data-processing rate and exibility that modern space missions require is causing an increase in the usage of onboard Field Programmable Gate Arrays (FPGAs). They provide the necessary exibility, allowing for the implementation of soft-processors and providing the re-programmable capabilities of the logic fabric. Besides, their development cost is notably lower than using ASICs because of the low volume of production inherent to space industry. All onboard components are exposed to the harsh space environment. That includes strong vibrations, a wide span of temperatures and the impact of ionizing particles. The latter is the one in which this work will focus. This section provides a brief introduction to FPGA technologies and the impacts of radiation in such devices, including previous research in emulating Single Event Upsets (SEUs).
This document focuses on SRAM-based FPGAs, particularly in the Kintex-Ultrascale KU060 FPGA from Xilinx, since the injection testing of the SpaceFibre is done on this device to provide the capabilities needed to implement high-throughput applications. Be-sides, Xilinx has recently released the space-grade (radiation-tolerant) version of the same architecture [11]. Therefore, it is expected to be the reference FPGA for future high-throughput applications. Later in this section, the architecture of this FPGA will be introduced as well as the most relevant results from previous radiation campaigns.
FPGA technologies
FPGAs are electronic circuits that implement a matrix of programmable logic, in the form of Look-up Tables (LUTs) that implement logic functions, and programmable intercon-nection blocks conceived to be programmed by the user (not during manufacture) to apply the desired function. Depending on the technology used to implement the programmable logic, there are three major technologies of FPGAs:
Anti-fuse FPGAs: use multiplex-based logic, and need the smallest switch inter-connection size. They incur the lowest power consumption and area as well. Despite being only one-time programmable, they are immune to the e ects of radiation, and still commonly used in really harsh environments, such as nuclear installations and spacecrafts or satellites [12]. However, they usually provide a small amount of equiv-alent logic gates, thus not being suitable for the most complex implementations and used only for the most safety-critical operations. Some examples are the RTAX series from Microsemi.
Flash-based FPGAs: re-programmable FPGAs for which the con guration mem-ory is implemented in a ash technology. The contents of the LUTs implementing logic functions, as well as the values that dictate how the programmable intercon-nect is wired, are stored in a ash con guration memory. Inherent to this technology is an almost non-existent susceptibility to single-event e ects. On top of that, the ash memory is non-volatile, and they do not need to be re-programmed after a re-boot. On the other hand, due to the technology used, re-programming a ash FPGA is more expensive in terms of power and time, and the technology node used is older and less e cient than their SRAM counterparts. Besides, their tolerance to the Total Ionizing Dose (TID) and Single-Event Latchup (SEL) is also an issue SRAM-based FPGAs: re-programmable FPGAs for which the con guration memory is implemented in SRAM technology, more precisely, Complementary metal-oxide-semiconductor (CMOS) Con guration Latches (CCLs) [14]. Even though the SRAM interconnect switches are the most expensive in terms of area [12], the usage of newer CMOS nodes fairly compensates it, allowing for higher scale integration, density and better e ciency than the ash-based FPGAs. Technology advantage makes their usage grow, since they are able to implement far more complex functions than their counterparts. Besides, they are less expensive to program, allowing for dynamic runtime partial recon guration (DPR) [15] used to modify the functional-ity of part of the FPGA while it is functioning. On the other hand, since the CMOS node they use is smaller, they have proven to be more susceptible to the e ects of radiation, as will be detailed in section 2.2.6. Furthermore, the volatile nature of SRAM technology cause the necessity of storing the programming bitstream in an external ash memory, introducing potential security breaches to the bitstream that are usually overcome with encryption.
KU060 FPGA structure
This section introduces the basic elements of the KU060 architecture, the SRAM FPGA used for the injection study. It is relevant to understand the elements that it’s composed of to better understand the e ects that radiation may cause.
The Ultrascale Architecture is focused in high-troughput applications, using a 20 nm tech-nology. Particularly, the Kintex Ultrascale architecture o ers an optimized performance per watt [16], including high-throughput Serializer-Deserializer (SerDes), dedicated Digi-tal Signal Processing (DSP) blocks, block ram memory, clock resources and Con gurable Logic Blocks (CLBs), the latter being the main building-block for general-purpose logic implementation. Each CLB, grouped physically in a slice, is formed by eight 6-input LUT (each of them con gurable as two 5-input LUT), 16 ip- ops, distributed memory and shift-register logic (allowing to implement distributed RAM in the SLICEM slice) as well as wide multiplexers to connect the elements of the CLB [14].
The elements within the FPGA are organized in columns. The most common structure is the one shown in gure 2.4, with one slice on each side of a routing switchbox, used to interconnect the di erent elements of the FPGA. One clock region is a tile formed by several columns of the same resource that span 60 rows of CLB columns, 24 DSP columns and 12 BRAM columns [17]. Clock regions are divided by horizontal and vertical clock routing and distribution tracks. Besides, the FPGAs also include a core column| integrating the System Monitor, con guration and PCIe blocks|as well as dedicated I/O banks, including high-performance GTH transceivers, I/O logic management and global clock bu ers. The device view o ered by Vivado, showing 4 clock regions in the KU060 device is depicted in gure 2.3
FPGA con guration
The con guration of the FPGA for ash and SRAM FPGAs is stored in the con gura-tion memory (CRAM) of the FPGA. It contains the logic tables for all LUTs, routing resources, interconnects, shift-registers, and ip- ops inside the FPGA, and is stored in a distributed fashion inside con guration latches or the distributed memory elements them-selves [18]. The other type of memory conforming a memory space in the FPGA is the Block RAM (BRAM) that provides the main accessible memory. It is utilized to store user data, and is stored in RAM blocks with several customization parameters.
The types of FPGAs mentioned before are con gured using a bitstream, that is, the bi-nary le that contains the con guration contents, along with all the necessary commands used to load it into the con guration memory of the device. For volatile SRAM FPGAs, support circuitry is included externally to launch the con guration automatically when the board is booted. For all Xilinx devices, including the targeted FPGA for this work, the minimum addressable unit of con guration is a frame, which in the Ultrascale tech-nology is composed of 123 words of 32 con guration bits. The total amount of frames composing the con guration memory is 37651 [19].
Each con guration bit a ects the functionality implemented by the FPGA in some way. Thus, if any value of con guration is changed, its functionality is modi ed. In order to obtain the con guration bits that are utilized by the circuit, Xilinx o ers the essential bits technology. The essential bits are de ned as the ones used to de ne the hardware in the FPGA, and can be automatically reported during the elaboration of the bitstream in the form of a mask le: the \.ebd » le. This le has the same number of bits|in the form of characters, one representing each bit|as the con guration memory, excluding the contents of the BRAM memories. Besides, the dynamic soft values stored in shift-registers and ip- ops are never marked as essential, since their values may change during execution. Thus, the ebd le marks the con guration bits that belong to the design and must remain static for the correct functioning of the circuit. For the sake of simplicity, these bits are denominated con guration bits for the rest of this work.
Unlike the BRAM and registered values, con guration bits are not easily accessible for the user. When the circuit is functioning, there are three ways to access con guration memory: through JTAG, which is extremely slow, through the SelectMAP port, that needs an external device connected to the selectMAP pins to manage the communication, and the Internal Con guration Access Port (ICAP), which can be accessed by the internal logic. Apart from the JTAG connectivity, which is used to program the FPGA in most evaluation boards in the initial stages of a design by using a PC, the most used interfaces are SelectMAP for programming from a ash memory, or using external scrubbing and ICAP for partial recon guration and internal scrubbing applications. Regardless of the speed and internal or external nature of the ports, similar commands are issued to read and write con guration memory through them. A major di erence is that, when using the ICAP port, all dynamic values stored in BRAM and registers are masked [19], so their values cannot be read. Besides, in the ebd le, these bits will never appear as essential. For more information on the Ultrascale family con guration, the reader is referred to [16].
Table of contents :
1 Introduction
1.1 Motivation
1.2 Purpose and goals
1.3 Research Questions
1.4 Research Methodology
2 Background
2.1 AMBA bus, GRLIB and LEON3
2.1.1 AMBA bus
2.1.2 LEON3
2.1.3 GRLIB
2.1.4 Additional tools
2.2 Radiation upsets on SRAM-based FPGAs
2.2.1 FPGA technologies
2.2.2 KU060 FPGA structure
2.2.3 FPGA conguration
2.2.4 Upset classication and occurrence
2.2.5 Radiation campaigns
2.2.6 Upset occurrence for the KU060 FPGA
2.2.7 Previous work in SEU injection
2.3 Space Fibre
2.3.1 Gigabit serial transceivers
2.3.2 Space Fibre modules
3 Firmware development for the
3.1 Board features and baseline
3.2 Developments carried out
3.2.1 DSU validation
3.2.2 Memory controllers validation
3.2.3 SPI controller and multiplexing validation
3.2.4 SerDes transceivers validation
3.2.5 ESA SpaceFibre IP integration and validation
3.2.6 STAR-Dundee SpaceFibre IP integration and validation
3.2.7 Final validation
4 SEU injection in the SpaceFibre IP core
4.1 Design choices
4.1.1 Limitations
4.2 Architecture of the injection experiment
4.2.1 DUT
4.2.2 Internal supervisor
4.2.3 Test controller
4.2.4 Injector
4.3 Testing framework
4.3.1 Addresses for the injections
4.3.2 Time between each injection
4.4 Software architecture
4.4.1 Software FSM
4.4.2 Serial Communication
4.5 Evaluated parameters
4.5.1 Severity of the failures
4.5.2 Application layer
4.5.3 Virtual channel layer
4.5.4 Retry layer
4.5.5 Lane layer
5 Results and discussion
5.1 Implementation results in the KU060
5.2 Results for the lane layer
5.2.1 Statistical validation
5.2.2 Other results
5.2.3 Discussion of the results for the lane layer
5.3 Results for the retry layer
5.3.1 Other results
5.3.2 Discussion
5.4 Results for the virtual channel layer
5.4.1 Other results
5.4.2 Discussion of the results
5.5 Results for the interface layer
5.5.1 Other results
5.5.2 Discussion of the results
5.6 Results for the broadcast layer
5.7 Results for the SpaceFibre IP core
5.7.1 Vulnerability factor
5.7.2 Discussion
5.8 Special events and limitations when using the SEM IP
5.8.1 ECC bits
5.8.2 Uncorrectable errors
5.8.3 Two errors caused by a single injection
5.9 Reliability of the SpaceFibre IP
6 Conclusion
6.1 Contributions
6.2 Future work
Bibliography
