Introduction: Integrating and deploying AI in Healthcare
VITEC was tasked with integrating and deploying AI models within an existing video processing device used in operating rooms. The device uses FPGA technology for video processing.
The trained AI models performed the following real-time assistance functions:
- Detection of the operation phase to inform and provide insights to the nursing team in real time, ensuring efficient workflow and patient safety in the operating room.
- Automating the detection and evacuation of impurities in the operating field to significantly help enhance the sterility and safety of surgical procedures
Data scientists at a medical laboratory developed and trained the AI models using their own available dataset, and VITEC’s FPGA designers helped integrate and deploy the model into a video processing device on an FPGA already used in the operating room. The data scientists used the FPGA AI Suite software flow to convert their trained AI model to FPGA AI inference IP and the FPGA designers used standard Quartus® Prime Software FPGA design flows to seamlessly integrate and validate the model and deploy it on an Arria® 10 FPGA.
Video has become central in the operating room, constantly offering surgeons new video-related assistance functions. The proliferation of video sources and monitors in operating theatres has led to increasingly powerful platforms with more inputs/outputs. Support for video signals in HDR format has revolutionized the quality of the images available to the surgeon during the operation and has led to the development of new real-time assistance functions.
Challenge: Cross-functional collaboration
One of the challenges is to integrate FPGA AI inference IP into an already existing video processing device. The video processing algorithm had to run in parallel with the AI inference IP. So, VITEC had to implement the AI inference IP alongside the current design without impacting the functionality and performance of both.
Once the AI inference IP had been integrated, the design had to be validated. The main challenge was setting up proper hand-off processes between the two teams to make sure the accuracy results seen by the data scientist team at the medical laboratory matched with what VITEC’s FPGA design team saw after integrating the inference IP into the FPGA.
Solution: FPGA AI Suite bridges the gap
FPGA AI Suite enables ease of use and push-button AI inference IP generation for Altera FPGA devices.
The data scientist team converted the trained AI model to FPGA AI inference IP using the OpenVINO™ open-source toolkit and FPGA AI Suite. OpenVINO is the frontend of FPGA AI Suite and helps convert AI models developed in any standard framework, such as PyTorch, TensorFlow, etc., into an intermediate representation that FPGA AI Suite uses to convert it to AI inference IP.
VITEC helped integrate the AI inference IP with the rest of the FPGA design using the Platform Designer system integration tool, closed timing in Quartus, and then validated the design.
Step 1. Build and train their AI model with popular frameworks.
Step 2. Use the OpenVINO toolkit to optimize the model and convert it to an Intermediate Representation (IR) data format.
Step 3. Use the FPGA AI Suite to run a quick FPGA performance and logic usage estimate to see if the model meets the target metrics.
Step 4. If step 3 meets the target metrics, use OpenVINO to run software emulation to check the accuracy of the FPGA AI Suite-generated IP.
Step 5. If step 3 does not meet the target metrics, use the auto-optimize feature within the FPGA AI Suite to change the FPGA IP architecture to achieve the desired performance target and logic usage. Iterate as necessary and proceed to step 4.
Note: Software emulation is currently only available for Agilex 5 and Agilex 3 FPGAs. The accuracy of other Altera FPGA families can be determined by running the design in hardware.
Referring to Figure 2, the VITEC engineers used the following steps to integrate the new AI inference IP with the existing FPGA image processing design.
Step 1. Obtain the FPGA AI Suite-generated IP from the medical laboratory data scientists.
Step 2. Use the Platform Designer tool within the Quartus Prime Software to integrate the AI inference IP with all other IP blocks and/or custom RTL logic. Use the typical process to finish the FPGA design (synthesis, place-and-route, close timing, finish simulations, analyze power, etc.) and generate the FPGA bitstream.
Step 3. Use the Quartus Prime Software programmer utility to program the FPGA-based hardware with the new AI-based design for further testing/validation.
The video processing device is controlled via an API to which AI inference IP’s control functions and commands have been added. The control functions include the ability to load the network, configure it, run inferences on images from the CPU, and retrieve the results. For the two applications, the AI networks were loaded via the API for real-time execution.
The Arria 10 FPGA device on which the video processing design is deployed had enough unused resources for the AI inference IPs to fit, making it easier to add the AI inference IP to the existing FPGA design. The data scientist team set the appropriate area targets to be used by the AI inference IP in the FPGA AI Suite tool and ran the architecture optimizer tool to obtain the best performance before handing it off to the FPGA team.
Integrating AI inference IP was just like integrating any other IP in Quartus. Once the FPGA team integrated the AI inference IP, the next step was validating the design. This involved checking that the AI IP was properly integrated and running inferences on it. The test plan also included multiple load/unload cycles, card initialization/deinitialization, and dozens of system start-ups/shutdowns to ensure the system was stable and reliable.
The data scientists provided labeled test images with the AI inference IP. The FPGA design team validated the execution of this data. This involved repeatedly running inferences on the test images and checking that the results always conformed to the table.
The integration test plan also had to ensure that the addition of the AI inference IP did not impact the existing image processing operations validated on the current FPGA design. The test plan was based on typical use cases.
Results: Successful deployment in the operating room
The application agility of Altera’s FPGAi solution enables engineers to craft and evolve AI solutions to stay at technology's cutting edge by using FPGA re-programmability, extended product lifecycles, and versatile I/O options.
VITEC successfully enhanced its video processing devices by adding powerful AI-powered features. Integrated into existing devices and workflows, these features are directly operational and significantly improve efficiencies during surgical procedures, adding value to existing FPGA-based platforms.
FPGA AI Suite’s single push-button AI inference IP flows seamlessly combine the software workflow used by data scientists and the hardware flow used by FPGA engineers into a generic end-to-end AI workflow, enabling efficient collaboration between them. Using Altera® FPGAs and the software flow with OpenVINO and FPGA AI Suite helps future-proof the designs and quickly adapt to new changes.