<div class='slidealt'>Experience kvm <a title='virtualization for embedded heterogeneous arm core platforms' href='/en/products'>virtualization extensions</a></div> <div class='slidealt'>Benefit from custom <a title='kvm on arm services full virtualization' href='/en/services'>virtualization services</a></div> <div class='slidealt'>KVM on ARMv7 and ARMv8 <a title='kvm-on-arm open source smu extensions' href='/en/solutions/guides/vfio-on-arm/'>IOMMU full virtualization</a></div> <div class='slidealt'>Virtualization research projects <a title='ARM multicore kvm open source' href='/en/research'>in cloud and embedded systems</a></div> <div class='slidealt'>Virtualization solutions for heterogeneous <a title='ARMv7-ARMv8 virtualization open source solutions' href='/en/solutions'>ARM multicore systems</a></div>

Virtual Open Systems Scientific Publications

SVFF: An Automated Framework for SR-IOV Virtual Function Management in FPGA Accelerated Virtualized Environments

SVFF - Framework for SR-IOV VF Management - conference CITS2023

Event

The 2023 International Conference on Computer, Information and Telecommunication Systems (CITS2023).

Keywords

Accelerators virtualization, FPGA, SR-IOV, Partial reconfiguration, VFIO, QEMU.

Authors

Stefano Cirici, Michele Paolino, Daniel Raho (Virtual Open Systems SAS).

Abstract

FPGA accelerator devices have emerged as a powerful platform for implementing high-performance and scalable solutions in a wide range of industries, leveraging their reconfigurability and virtualization capabilities. Virtualization, in particular, offers several benefits including improved security by resource isolation and sharing, and SR-IOV is the main solution for enabling it on FPGAs. This paper introduces the SR-IOV Virtual Function Framework (SVFF), a solution that aims to simplify and enhance the management of Virtual Functions (VFs) on PCIe-attached FPGA devices in Linux and QEMU/KVM environments, solving the lack of SR-IOV re-configuration support on guests. The framework leverages the SR-IOV support in the Xilinx Queue-based Direct Memory Access (QDMA) to automate the creation, attachment, detachment, and reconfiguration of VFs to different Virtual Machines (VMs). A novel pause functionality for the VFIO device has been implemented in QEMU to enable the detachment of VFs from the host without detaching them from the guest, making reconfiguration of VFs transparent for guests that already have a VF attached to them without any performance loss. The proposed solution offers the ability to automatically and seamlessly assign a set of VFs to different VMs and adjust the configuration on the fly. Thanks to the pause functionality, it also offers the ability to attach additional VFs to new VMs without affecting devices already attached to other VMs.

Introduction

In recent years, the widespread adoption of virtualization technologies has led to a growing demand for effective and efficient management of hardware resources in data centers, cloud computing, and high-performance computing environments. The management of devices in these environments can be challenging, especially when dealing with multiple VMs and frequent reconfigurations. To accelerate complex workloads, Field Programmable Gate Arrays (FPGAs) are more and more used to provide a fast-deployment, fast-prototyping solution to custom problems, with an increasing interest in Artificial Intelligence (AI) and Machine Learning (ML) domains. Virtualization of FPGA resources also enables scenarios where a single FPGA can provide multiple devices to different VMs, to enhance the workload and security.

Addressing some of the VMs' guest acceleration is already possible by using the Single Root I/O Virtualization (SR-IOV) technology to allow multiple VMs to share the same physical device providing direct access to the hardware with minimal to no overhead. However, SR-IOV usage is not always straightforward and can lead to complex scenarios, especially when there is the need to change the VF configuration on the fly which requires the removal of every VF from each VM. This is not a problem in applications with fixed and predictable workloads or with dedicated resources for the VMs, but in environments such as cloud computing, software as a service, high-performance computing, and edge computing where workloads can vary rapidly, resources need to be shared among different VMs, and devices need to be reconfigured and reassigned, the limitations of SR-IOV become more evident. In these cases, a solution that allows automatic assignment and reconfiguration of virtual functions, can significantly simplify device management and improve system efficiency.

Access the full content of this publication

Login or register to access full information