Heterogeneous Accelerated Compute Cluster (HACC) at NUS

Workshop
User Guide
Booking System

The Heterogeneous Accelerated Compute Cluster (HACC) program is a special initiative to support novel research in adaptive compute acceleration for high performance computing (HPC).

User Guide of HACC Cluster at NUS

I. Registration

For researchers outside NUS, please request access through Xilinx University Program.

For researchers at NUS, please apply through the following link: NUS HACC Account Registration.

The user account expires after one year or there is no login event within two months. You will get a notification from NUS of this event one month in advance. If you plan to keep using the resources, please follow the instruction in the notification mail.

II. Compute Node Configuration

We have the following FPGA boards.

OS: Ubuntu 20.04.4 (Linux 5.4)

Hardware Setup
Shell/Arch Version
XRT Version
Toolchain Version
GPU/FPGA Cluster
VCK5000 x2
xilinx_vck5000_gen4x8_qdma_2_202220_1
2.14.384
Vitis 2022.2
U55C x6
xilinx_u55c_gen3x16_xdma_3_202210_1
2.14.384
Vitis 2022.2
Alveo U250 x1
xilinx_u250_gen3x16_xdma_4_1_202210_1
2.14.384
Vitis 2022.2
MI210 x12
gfx90a
-
ROCm-5.4.2
U250 Cluster
Alveo U250 x4
xilinx_u250_gen3x16_xdma_3_1_202020_1
2.14.354
Vitis 2021.2
U280 Cluster
Alveo U280 x4
xilinx_u280_xdma_201920_3
2.14.354
Vitis 2021.2
VCK5000 Cluster
VCK5000 x4
xilinx_vck5000_gen4x8_qdma_2_202220_1
2.14.384
Vitis 2022.2
U50 Single Node
Alveo U50 x1
xilinx_u50_gen3x16_xdma_201920_3
2.14.354
Vitis 2021.2
GPU Cluster
MI25 x8
gfx906
-
ROCm-5.4.2
MI100 x16
gfx908
-
ROCm-5.4.2
    10 10
  • 1
    • 10

III. Usage

a) Access the cluster

Access through SSH:

ssh username@xacchead.d2.comp.nus.edu.sg

Please following the instructions in the email that contains your account’s information to set up the 2FA authentication.

b) Setup FPGA compilation environment

XACC clusters now only support the Vitis Design Flow. Benefiting from XRT environments and C/C++ based HLS tools, FPGA developlors can easy test their code in real hardware, and algorithm designers can ignore the hardware details. If you want to evaluate some RTL modules, we suggest you try the mix programming with Vitis following this and examples.

Users should then use the following command to setup the environment:

# Set XRT Environment
source /opt/xilinx/xrt/setup.sh
# Set Vitis Environment, for the node installed Vitis 2021.2
source  /opt/Xilinx/Vitis/2021.2/settings64.sh

c) Run FPGA accelerated tasks with Slurm

Please following this on how to write a Slurm job to utilize FPGAs.

d) Use the shared storage

/data is a shared path in all compute nodes for users to store insensitive data. Please keep your data size within 50GB.

e) Setup VNC for the visualization of waveform simulation

We have installed VNC server in our cluster for a better experience on waveform-based FPGA simulation. To enable it, please follow this to setup the VNC service for your account.

IV. Support

Cluster environment:

If you need help (e.g., software installation), you can submit issues in here, or send email to hacc@comp.nus.edu.sg

Password reset:

In case you forgot your account’s password, you can email hacc@comp.nus.edu.sg with the following information

Subject: Reset Password

Context: Account: your_username

V. Acknowledgement

We would like to thank Xilinx for the hardware donation.

HACC acknowledgment

AMD encourages HACC users to acknowledge AMD’s support of your research in your presentations, papers, posters, and press releases. A brief statement such as the one below is recommended.

Example of acknowledgment

This work was supported in part by AMD under the Heterogeneous Accelerated Compute Clusters (HACC) program (formerly known as the XACC program - Xilinx Adaptive Compute Cluster program).