{"id":1000,"date":"2019-05-30T16:45:11","date_gmt":"2019-05-30T16:45:11","guid":{"rendered":"http:\/\/stage.wizzdev.pl\/?p=1000"},"modified":"2023-05-26T13:52:31","modified_gmt":"2023-05-26T13:52:31","slug":"fpga_data_transfer_demo_1","status":"publish","type":"post","link":"https:\/\/stage2.wizzdev.pl\/blog\/fpga_data_transfer_demo_1\/","title":{"rendered":"FPGA Data Transfer demo #1"},"content":{"rendered":"<p>This is the <em>first<\/em> post of a short tutorial series aiming to demonstrate the capabilities of high-speed data transfer using FPGA based platform and an elementary Python application.<\/p>\n<p><strong>FPGA Data Transfer demo<\/strong> is a simple and exemplary project designed for high-speed data acquisition. It integrates Opal Kelly XEM7310 platform (with Xilinx Artix-7 FPGA) as a data generation module and a Python application on a PC (connected via USB3.0) that receives and processes the data. The data is then saved to a file in <strong>HDF5<\/strong> format which has been designed for storing and managing large amounts of data (such as time-series) in a hierarchical binary format.<\/p>\n<p>A proposed system with a simple GUI is capable of:<\/p>\n<ul>\n<li>controlling data acquisition<\/li>\n<li>writing whole data to HDF5 file<\/li>\n<li>live data plotting<\/li>\n<\/ul>\n<p>&nbsp;<\/p>\n<h2>Overview<\/h2>\n<p>In this introductory post, we will cover all the required hardware and software elements that will be used within this project. After everything is set up the user will be able to run an example script to check the communication with the board and the possibilities of current hardware setup.<\/p>\n<p>&nbsp;<\/p>\n<h2>What is needed?<\/h2>\n<p><strong>Hardware<\/strong><\/p>\n<ul>\n<li><a href=\"https:\/\/opalkelly.com\/products\/xem7310\/\">Opal Kelly XEM7130 board<\/a>\u00a0 (we used XEM7310-A75 version)<\/li>\n<\/ul>\n<p><strong>Software<\/strong><\/p>\n<ul>\n<li style=\"list-style-type: none;\">\n<ul>\n<li>Opal Kelly FrontPanel SDK (including HDL files and Python API) &#8211; available from Opal Kelly&#8217;s <a href=\"https:\/\/pins.opalkelly.com\/downloads\" rel=\"noopener noreferrer\">Pins portal<\/a> to approved registered customers,<\/li>\n<li>Xilinx Vivado Design Suite &#8211; (for compiling VHDL code and creating .bit files to program the FPGA). WebPack edition is available for free from <a href=\"https:\/\/www.xilinx.com\/support\/download.html\" rel=\"noopener noreferrer\">Xilinx website,<\/a><\/li>\n<\/ul>\n<\/li>\n<\/ul>\n<p>(registration required, use Web installer like Xilinx recommends, also you may want to deselect other boards to save some disc space . Resulting required disc space should be around 37GB.<\/p>\n<ul>\n<li>Python interpreter (Opal Kelly Python API version should match used interpreter),<\/li>\n<li>Python IDE of choice\u00a0 <em>(optional)<\/em><\/li>\n<\/ul>\n<p><em>This tutorial was tested on Linux Ubuntu 18.04, with Python 3.6<\/em><\/p>\n<p>Additional files needed to perform the test described later in this post:<\/p>\n<ul>\n<li><em>PipeTest.bit. &#8211;<\/em> generated sample bitfile available from the <a href=\"https:\/\/pins.opalkelly.com\/downloads\" rel=\"noopener noreferrer\">Opal Kelly website<\/a> (look for \u201cXEM7310-A75 Sample Bitfiles\u201d),<\/li>\n<li>pipe_test.py &#8211; script from our <a href=\"https:\/\/github.com\/wizzdev-pl\/fpga_data_transfer_demo\" rel=\"noopener noreferrer\">GitHub repository<\/a> \u00a0(python\/tools\/pipe_test.py).<\/li>\n<\/ul>\n<p>We recommend cloning or downloading our repository to maintain the file structure.<\/p>\n<p><strong>Note:<\/strong> Before the first run copy your FrontPanel Python API files (<em>_ok.so <\/em>and<em> ok.py<\/em>) into this folder:<\/p>\n<p><span class=\"js-repo-root text-bold\"><span class=\"js-path-segment\">\u00a0\u00a0\u00a0\u00a0 fpga_data_transfer_demo<\/span><\/span><span class=\"separator\">\/<\/span><span class=\"js-path-segment\">python<\/span><span class=\"separator\">\/<\/span><strong class=\"final-path\">FrontPanelAPI<\/strong><span class=\"separator\">\/<\/span><\/p>\n<p>&nbsp;<\/p>\n<h2>Checking the transfer capabilities<\/h2>\n<p>It&#8217;s good to know what maximum transfer rate is achievable on your setup and verify whether the application can cope with desired data throughput at the very beginning. To send data we will use Opal Kelly&#8217;s provided bulk transfers called Pipes (more info about Opal Kelly Pipes can be found <a href=\"https:\/\/docs.opalkelly.com\/display\/FPSDK\/Performance\" rel=\"noopener noreferrer\">here<\/a>). Transfer rate strongly depends on used buffer and block (for Block-Throttled Pipes) lengths due to the existence of overhead. Usually, the speed increases with buffer length. Its upper limits are also determined by USB hardware capabilities.<br \/>\nTo run the test:<\/p>\n<ol>\n<li>power up the board,<\/li>\n<li>plug the USB cable to the computer,<\/li>\n<li>go to the directory where <em>pipe_test.py<\/em> resides and run:<\/li>\n<\/ol>\n<pre> \u00a0\u00a0 python3 pipe_test.py path_to_bitfile\/PipeTest.bit<\/pre>\n<p>(remember to change path_to_bitfile to an actual path on your computer!)<\/p>\n<p>The script will perform transfer speed tests for various transfer lengths and buffers. It will print the result to the console. A certain pair of BLOCK_LENGTH and TRANSFER_LENGTH (in bytes) can be also tested by running:<\/p>\n<pre> \u00a0\u00a0 python3 pipe_test.py path_to_bit_file <strong>pipes<\/strong> <strong>BLOCK_LENGTH<\/strong> <strong>TRANSFER_LENGTH\r\n<\/strong><\/pre>\n<p>For our setup, with SuperSpeed USB3.0 interface we were able to observe transfer speed up to almost 300MiB\/s using Python. If maximum transfer lengths do not exceed 50MB\/s it may indicate that USB2.0 port is used.<\/p>\n<p>Bear in mind that 340 MiB\/s transfer speed advertised on Opal Kelly&#8217;s website was probably measured using C++ application, and in fact, we achieved similar results using their PipeTest.cpp benchmark.\u00a0 Also, these results are taken without any data processing, and the final result for your application may be lower, but hey &#8211;\u00a0 its Python!<\/p>\n<p>&nbsp;<\/p>\n<h2>What&#8217;s next?<\/h2>\n<p>In the next posts we will be:<\/p>\n<p>&#8211; setting up a project in Vivado and creating a simple FPGA demo application in VHDL,<\/p>\n<p>&#8211; writing a Python application to read the data and save them to HDF5 file<\/p>\n<p>&#8211; visualizing the data in Python<\/p>\n<p>&nbsp;<\/p>\n<p>We hope you will find our demo project helpful, and in case of any questions feel free to leave a comment!<\/p>\n<p>&nbsp;<\/p>\n<div id=\"simple-translate\">\n<div>\n<div class=\"simple-translate-button\" style=\"height: 22px; width: 22px;\"><\/div>\n<div class=\"simple-translate-panel\" style=\"width: 300px; height: 200px; font-size: 13px; background-color: #ffffff;\">\n<div class=\"simple-translate-result-wrapper\" style=\"overflow: hidden;\">\n<p class=\"simple-translate-candidate\" style=\"color: #737373;\">\n<\/div>\n<\/div>\n<\/div>\n<\/div>\n","protected":false},"excerpt":{"rendered":"<p>This is the first post of a short tutorial series aiming to demonstrate the capabilities of high-speed data transfer using FPGA based platform and an elementary Python application. FPGA Data Transfer <a class=\"read-more\" href=\"https:\/\/stage2.wizzdev.pl\/blog\/fpga_data_transfer_demo_1\/\">\u00a0[&#8230;]<\/a><\/p>\n","protected":false},"author":1,"featured_media":2657,"comment_status":"open","ping_status":"open","sticky":false,"template":"","format":"standard","meta":{"content-type":""},"categories":[7],"tags":[6,8,9,10],"acf":[],"yoast_head":"<!-- This site is optimized with the Yoast SEO plugin v20.8 - https:\/\/yoast.com\/wordpress\/plugins\/seo\/ -->\n<title>FPGA Data Transfer demo #1 - WizzDev<\/title>\n<meta name=\"robots\" content=\"index, follow, max-snippet:-1, max-image-preview:large, max-video-preview:-1\" \/>\n<link rel=\"canonical\" href=\"https:\/\/stage2.wizzdev.pl\/blog\/fpga_data_transfer_demo_1\/\" \/>\n<meta property=\"og:locale\" content=\"en_US\" \/>\n<meta property=\"og:type\" content=\"article\" \/>\n<meta property=\"og:title\" content=\"FPGA Data Transfer demo #1 - WizzDev\" \/>\n<meta property=\"og:description\" content=\"This is the first post of a short tutorial series aiming to demonstrate the capabilities of high-speed data transfer using FPGA based platform and an elementary Python application. FPGA Data Transfer \u00a0[...]\" \/>\n<meta property=\"og:url\" content=\"https:\/\/stage2.wizzdev.pl\/blog\/fpga_data_transfer_demo_1\/\" \/>\n<meta property=\"og:site_name\" content=\"WizzDev\" \/>\n<meta property=\"article:published_time\" content=\"2019-05-30T16:45:11+00:00\" \/>\n<meta property=\"article:modified_time\" content=\"2023-05-26T13:52:31+00:00\" \/>\n<meta property=\"og:image\" content=\"https:\/\/stage2.wizzdev.pl\/wp-content\/uploads\/2022\/03\/hintergrund-1542000578aJX-e1559906287202-690x248-1.jpg\" \/>\n\t<meta property=\"og:image:width\" content=\"690\" \/>\n\t<meta property=\"og:image:height\" content=\"248\" \/>\n\t<meta property=\"og:image:type\" content=\"image\/jpeg\" \/>\n<meta name=\"author\" content=\"admin\" \/>\n<meta name=\"twitter:card\" content=\"summary_large_image\" \/>\n<meta name=\"twitter:label1\" content=\"Written by\" \/>\n\t<meta name=\"twitter:data1\" content=\"admin\" \/>\n\t<meta name=\"twitter:label2\" content=\"Est. reading time\" \/>\n\t<meta name=\"twitter:data2\" content=\"4 minutes\" \/>\n<script type=\"application\/ld+json\" class=\"yoast-schema-graph\">{\"@context\":\"https:\/\/schema.org\",\"@graph\":[{\"@type\":\"Article\",\"@id\":\"https:\/\/stage2.wizzdev.pl\/blog\/fpga_data_transfer_demo_1\/#article\",\"isPartOf\":{\"@id\":\"https:\/\/stage2.wizzdev.pl\/blog\/fpga_data_transfer_demo_1\/\"},\"author\":{\"name\":\"admin\",\"@id\":\"https:\/\/stage2.wizzdev.pl\/#\/schema\/person\/a82672ac0e310ffa148007ec229c09cf\"},\"headline\":\"FPGA Data Transfer demo #1\",\"datePublished\":\"2019-05-30T16:45:11+00:00\",\"dateModified\":\"2023-05-26T13:52:31+00:00\",\"mainEntityOfPage\":{\"@id\":\"https:\/\/stage2.wizzdev.pl\/blog\/fpga_data_transfer_demo_1\/\"},\"wordCount\":712,\"commentCount\":0,\"publisher\":{\"@id\":\"https:\/\/stage2.wizzdev.pl\/#organization\"},\"keywords\":[\"FPGA\",\"HDF5\",\"Python\",\"TIme Series Data\"],\"articleSection\":[\"FPGA projects\"],\"inLanguage\":\"en\",\"potentialAction\":[{\"@type\":\"CommentAction\",\"name\":\"Comment\",\"target\":[\"https:\/\/stage2.wizzdev.pl\/blog\/fpga_data_transfer_demo_1\/#respond\"]}]},{\"@type\":\"WebPage\",\"@id\":\"https:\/\/stage2.wizzdev.pl\/blog\/fpga_data_transfer_demo_1\/\",\"url\":\"https:\/\/stage2.wizzdev.pl\/blog\/fpga_data_transfer_demo_1\/\",\"name\":\"FPGA Data Transfer demo #1 - WizzDev\",\"isPartOf\":{\"@id\":\"https:\/\/stage2.wizzdev.pl\/#website\"},\"datePublished\":\"2019-05-30T16:45:11+00:00\",\"dateModified\":\"2023-05-26T13:52:31+00:00\",\"breadcrumb\":{\"@id\":\"https:\/\/stage2.wizzdev.pl\/blog\/fpga_data_transfer_demo_1\/#breadcrumb\"},\"inLanguage\":\"en\",\"potentialAction\":[{\"@type\":\"ReadAction\",\"target\":[\"https:\/\/stage2.wizzdev.pl\/blog\/fpga_data_transfer_demo_1\/\"]}]},{\"@type\":\"BreadcrumbList\",\"@id\":\"https:\/\/stage2.wizzdev.pl\/blog\/fpga_data_transfer_demo_1\/#breadcrumb\",\"itemListElement\":[{\"@type\":\"ListItem\",\"position\":1,\"name\":\"Home\",\"item\":\"https:\/\/stage2.wizzdev.pl\/\"},{\"@type\":\"ListItem\",\"position\":2,\"name\":\"FPGA Data Transfer demo #1\"}]},{\"@type\":\"WebSite\",\"@id\":\"https:\/\/stage2.wizzdev.pl\/#website\",\"url\":\"https:\/\/stage2.wizzdev.pl\/\",\"name\":\"WizzDev\",\"description\":\"\",\"publisher\":{\"@id\":\"https:\/\/stage2.wizzdev.pl\/#organization\"},\"potentialAction\":[{\"@type\":\"SearchAction\",\"target\":{\"@type\":\"EntryPoint\",\"urlTemplate\":\"https:\/\/stage2.wizzdev.pl\/?s={search_term_string}\"},\"query-input\":\"required name=search_term_string\"}],\"inLanguage\":\"en\"},{\"@type\":\"Organization\",\"@id\":\"https:\/\/stage2.wizzdev.pl\/#organization\",\"name\":\"WizzDev\",\"url\":\"https:\/\/stage2.wizzdev.pl\/\",\"logo\":{\"@type\":\"ImageObject\",\"inLanguage\":\"en\",\"@id\":\"https:\/\/stage2.wizzdev.pl\/#\/schema\/logo\/image\/\",\"url\":\"https:\/\/stage2.wizzdev.pl\/wp-content\/uploads\/2022\/02\/Logo-WizzDev-wersja-1-PNG-granat-niebieski-1-1.jpg\",\"contentUrl\":\"https:\/\/stage2.wizzdev.pl\/wp-content\/uploads\/2022\/02\/Logo-WizzDev-wersja-1-PNG-granat-niebieski-1-1.jpg\",\"width\":179,\"height\":47,\"caption\":\"WizzDev\"},\"image\":{\"@id\":\"https:\/\/stage2.wizzdev.pl\/#\/schema\/logo\/image\/\"},\"sameAs\":[\"https:\/\/twitter.com\/wizz_dev\",\"https:\/\/www.upwork.com\/ag\/wizzdev\/\",\"https:\/\/github.com\/wizzdev-pl\",\"https:\/\/linkedin.com\/company\/wizzdev\"]},{\"@type\":\"Person\",\"@id\":\"https:\/\/stage2.wizzdev.pl\/#\/schema\/person\/a82672ac0e310ffa148007ec229c09cf\",\"name\":\"admin\",\"image\":{\"@type\":\"ImageObject\",\"inLanguage\":\"en\",\"@id\":\"https:\/\/stage2.wizzdev.pl\/#\/schema\/person\/image\/\",\"url\":\"https:\/\/secure.gravatar.com\/avatar\/a040dcbe510bab2c1867895ac03435c7?s=96&d=mm&r=g\",\"contentUrl\":\"https:\/\/secure.gravatar.com\/avatar\/a040dcbe510bab2c1867895ac03435c7?s=96&d=mm&r=g\",\"caption\":\"admin\"},\"sameAs\":[\"https:\/\/stage2.wizzdev.pl\"],\"url\":\"https:\/\/stage2.wizzdev.pl\/blog\/author\/admin\/\"}]}<\/script>\n<!-- \/ Yoast SEO plugin. -->","yoast_head_json":{"title":"FPGA Data Transfer demo #1 - WizzDev","robots":{"index":"index","follow":"follow","max-snippet":"max-snippet:-1","max-image-preview":"max-image-preview:large","max-video-preview":"max-video-preview:-1"},"canonical":"https:\/\/stage2.wizzdev.pl\/blog\/fpga_data_transfer_demo_1\/","og_locale":"en_US","og_type":"article","og_title":"FPGA Data Transfer demo #1 - WizzDev","og_description":"This is the first post of a short tutorial series aiming to demonstrate the capabilities of high-speed data transfer using FPGA based platform and an elementary Python application. FPGA Data Transfer \u00a0[...]","og_url":"https:\/\/stage2.wizzdev.pl\/blog\/fpga_data_transfer_demo_1\/","og_site_name":"WizzDev","article_published_time":"2019-05-30T16:45:11+00:00","article_modified_time":"2023-05-26T13:52:31+00:00","og_image":[{"width":690,"height":248,"url":"https:\/\/stage2.wizzdev.pl\/wp-content\/uploads\/2022\/03\/hintergrund-1542000578aJX-e1559906287202-690x248-1.jpg","type":"image\/jpeg"}],"author":"admin","twitter_card":"summary_large_image","twitter_misc":{"Written by":"admin","Est. reading time":"4 minutes"},"schema":{"@context":"https:\/\/schema.org","@graph":[{"@type":"Article","@id":"https:\/\/stage2.wizzdev.pl\/blog\/fpga_data_transfer_demo_1\/#article","isPartOf":{"@id":"https:\/\/stage2.wizzdev.pl\/blog\/fpga_data_transfer_demo_1\/"},"author":{"name":"admin","@id":"https:\/\/stage2.wizzdev.pl\/#\/schema\/person\/a82672ac0e310ffa148007ec229c09cf"},"headline":"FPGA Data Transfer demo #1","datePublished":"2019-05-30T16:45:11+00:00","dateModified":"2023-05-26T13:52:31+00:00","mainEntityOfPage":{"@id":"https:\/\/stage2.wizzdev.pl\/blog\/fpga_data_transfer_demo_1\/"},"wordCount":712,"commentCount":0,"publisher":{"@id":"https:\/\/stage2.wizzdev.pl\/#organization"},"keywords":["FPGA","HDF5","Python","TIme Series Data"],"articleSection":["FPGA projects"],"inLanguage":"en","potentialAction":[{"@type":"CommentAction","name":"Comment","target":["https:\/\/stage2.wizzdev.pl\/blog\/fpga_data_transfer_demo_1\/#respond"]}]},{"@type":"WebPage","@id":"https:\/\/stage2.wizzdev.pl\/blog\/fpga_data_transfer_demo_1\/","url":"https:\/\/stage2.wizzdev.pl\/blog\/fpga_data_transfer_demo_1\/","name":"FPGA Data Transfer demo #1 - WizzDev","isPartOf":{"@id":"https:\/\/stage2.wizzdev.pl\/#website"},"datePublished":"2019-05-30T16:45:11+00:00","dateModified":"2023-05-26T13:52:31+00:00","breadcrumb":{"@id":"https:\/\/stage2.wizzdev.pl\/blog\/fpga_data_transfer_demo_1\/#breadcrumb"},"inLanguage":"en","potentialAction":[{"@type":"ReadAction","target":["https:\/\/stage2.wizzdev.pl\/blog\/fpga_data_transfer_demo_1\/"]}]},{"@type":"BreadcrumbList","@id":"https:\/\/stage2.wizzdev.pl\/blog\/fpga_data_transfer_demo_1\/#breadcrumb","itemListElement":[{"@type":"ListItem","position":1,"name":"Home","item":"https:\/\/stage2.wizzdev.pl\/"},{"@type":"ListItem","position":2,"name":"FPGA Data Transfer demo #1"}]},{"@type":"WebSite","@id":"https:\/\/stage2.wizzdev.pl\/#website","url":"https:\/\/stage2.wizzdev.pl\/","name":"WizzDev","description":"","publisher":{"@id":"https:\/\/stage2.wizzdev.pl\/#organization"},"potentialAction":[{"@type":"SearchAction","target":{"@type":"EntryPoint","urlTemplate":"https:\/\/stage2.wizzdev.pl\/?s={search_term_string}"},"query-input":"required name=search_term_string"}],"inLanguage":"en"},{"@type":"Organization","@id":"https:\/\/stage2.wizzdev.pl\/#organization","name":"WizzDev","url":"https:\/\/stage2.wizzdev.pl\/","logo":{"@type":"ImageObject","inLanguage":"en","@id":"https:\/\/stage2.wizzdev.pl\/#\/schema\/logo\/image\/","url":"https:\/\/stage2.wizzdev.pl\/wp-content\/uploads\/2022\/02\/Logo-WizzDev-wersja-1-PNG-granat-niebieski-1-1.jpg","contentUrl":"https:\/\/stage2.wizzdev.pl\/wp-content\/uploads\/2022\/02\/Logo-WizzDev-wersja-1-PNG-granat-niebieski-1-1.jpg","width":179,"height":47,"caption":"WizzDev"},"image":{"@id":"https:\/\/stage2.wizzdev.pl\/#\/schema\/logo\/image\/"},"sameAs":["https:\/\/twitter.com\/wizz_dev","https:\/\/www.upwork.com\/ag\/wizzdev\/","https:\/\/github.com\/wizzdev-pl","https:\/\/linkedin.com\/company\/wizzdev"]},{"@type":"Person","@id":"https:\/\/stage2.wizzdev.pl\/#\/schema\/person\/a82672ac0e310ffa148007ec229c09cf","name":"admin","image":{"@type":"ImageObject","inLanguage":"en","@id":"https:\/\/stage2.wizzdev.pl\/#\/schema\/person\/image\/","url":"https:\/\/secure.gravatar.com\/avatar\/a040dcbe510bab2c1867895ac03435c7?s=96&d=mm&r=g","contentUrl":"https:\/\/secure.gravatar.com\/avatar\/a040dcbe510bab2c1867895ac03435c7?s=96&d=mm&r=g","caption":"admin"},"sameAs":["https:\/\/stage2.wizzdev.pl"],"url":"https:\/\/stage2.wizzdev.pl\/blog\/author\/admin\/"}]}},"_links":{"self":[{"href":"https:\/\/stage2.wizzdev.pl\/wp-json\/wp\/v2\/posts\/1000"}],"collection":[{"href":"https:\/\/stage2.wizzdev.pl\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/stage2.wizzdev.pl\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/stage2.wizzdev.pl\/wp-json\/wp\/v2\/users\/1"}],"replies":[{"embeddable":true,"href":"https:\/\/stage2.wizzdev.pl\/wp-json\/wp\/v2\/comments?post=1000"}],"version-history":[{"count":4,"href":"https:\/\/stage2.wizzdev.pl\/wp-json\/wp\/v2\/posts\/1000\/revisions"}],"predecessor-version":[{"id":4758,"href":"https:\/\/stage2.wizzdev.pl\/wp-json\/wp\/v2\/posts\/1000\/revisions\/4758"}],"wp:featuredmedia":[{"embeddable":true,"href":"https:\/\/stage2.wizzdev.pl\/wp-json\/wp\/v2\/media\/2657"}],"wp:attachment":[{"href":"https:\/\/stage2.wizzdev.pl\/wp-json\/wp\/v2\/media?parent=1000"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/stage2.wizzdev.pl\/wp-json\/wp\/v2\/categories?post=1000"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/stage2.wizzdev.pl\/wp-json\/wp\/v2\/tags?post=1000"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}