Back to blog

FPGA Data Transfer demo #1

This is the first post of a short tutorial series aiming to demonstrate the capabilities of high-speed data transfer using FPGA based platform and an elementary Python application.

FPGA Data Transfer demo is a simple and exemplary project designed for high-speed data acquisition. It integrates Opal Kelly XEM7310 platform (with Xilinx Artix-7 FPGA) as a data generation module and a Python application on a PC (connected via USB3.0) that receives and processes the data. The data is then saved to a file in HDF5 format which has been designed for storing and managing large amounts of data (such as time-series) in a hierarchical binary format.

A proposed system with a simple GUI is capable of:

controlling data acquisition
writing whole data to HDF5 file
live data plotting

Overview

In this introductory post, we will cover all the required hardware and software elements that will be used within this project. After everything is set up the user will be able to run an example script to check the communication with the board and the possibilities of current hardware setup.

What is needed?

Hardware

Opal Kelly XEM7130 board (we used XEM7310-A75 version)

Software

- Opal Kelly FrontPanel SDK (including HDL files and Python API) – available from Opal Kelly’s Pins portal to approved registered customers,
- Xilinx Vivado Design Suite – (for compiling VHDL code and creating .bit files to program the FPGA). WebPack edition is available for free from Xilinx website,

(registration required, use Web installer like Xilinx recommends, also you may want to deselect other boards to save some disc space . Resulting required disc space should be around 37GB.

Python interpreter (Opal Kelly Python API version should match used interpreter),
Python IDE of choice (optional)

This tutorial was tested on Linux Ubuntu 18.04, with Python 3.6

Additional files needed to perform the test described later in this post:

PipeTest.bit. – generated sample bitfile available from the Opal Kelly website (look for “XEM7310-A75 Sample Bitfiles”),
pipe_test.py – script from our GitHub repository (python/tools/pipe_test.py).

We recommend cloning or downloading our repository to maintain the file structure.

Note: Before the first run copy your FrontPanel Python API files (_ok.so and ok.py) into this folder:

fpga_data_transfer_demo/python/FrontPanelAPI/

Checking the transfer capabilities

It’s good to know what maximum transfer rate is achievable on your setup and verify whether the application can cope with desired data throughput at the very beginning. To send data we will use Opal Kelly’s provided bulk transfers called Pipes (more info about Opal Kelly Pipes can be found here). Transfer rate strongly depends on used buffer and block (for Block-Throttled Pipes) lengths due to the existence of overhead. Usually, the speed increases with buffer length. Its upper limits are also determined by USB hardware capabilities.
To run the test:

power up the board,
plug the USB cable to the computer,
go to the directory where pipe_test.py resides and run:

    python3 pipe_test.py path_to_bitfile/PipeTest.bit

(remember to change path_to_bitfile to an actual path on your computer!)

The script will perform transfer speed tests for various transfer lengths and buffers. It will print the result to the console. A certain pair of BLOCK_LENGTH and TRANSFER_LENGTH (in bytes) can be also tested by running:

    python3 pipe_test.py path_to_bit_file pipes BLOCK_LENGTH TRANSFER_LENGTH

For our setup, with SuperSpeed USB3.0 interface we were able to observe transfer speed up to almost 300MiB/s using Python. If maximum transfer lengths do not exceed 50MB/s it may indicate that USB2.0 port is used.

Bear in mind that 340 MiB/s transfer speed advertised on Opal Kelly’s website was probably measured using C++ application, and in fact, we achieved similar results using their PipeTest.cpp benchmark. Also, these results are taken without any data processing, and the final result for your application may be lower, but hey – its Python!