arXiv:2604.09240v1 [cs.LG] 10 Apr 2026
D IFF HLS: Differential Learning for High-Level Synthesis QoR Prediction with GNNs and LLM Code Embeddings Zedong Peng
Zeju Li
Qiang Xu
Shanghai Jiao Tong University Shanghai, China [email protected]
The Chinese University of Hong Kong Hong Kong, China [email protected]
The Chinese University of Hong Kong Hong Kong, China [email protected]
Jieru Zhao Shanghai Jiao Tong University Shanghai, China [email protected]
Abstract—High-Level Synthesis (HLS) compiles C/C++ into RTL, but exploring pragma-driven optimization choices remains expensive because each design point requires time-consuming synthesis. We propose D IFF HLS, a differential learning framework for HLS Quality-of-Result (QoR) prediction that learns from kernel–design pairs: a kernel baseline and a pragma-inserted design variant. D IFF HLS encodes kernel and design intermediaterepresentation graphs with dedicated graph neural network (GNN) branches, and augments the delta pathway with code embeddings from a pretrained code large language model (LLM). Instead of regressing absolute targets directly, we jointly predict the kernel baseline and the design-induced delta, and compose them to obtain the design prediction. On PolyBench, D IFF HLS attains lower average MAPE than GNN baselines under four GNN backbones, and LLM code embeddings consistently improve over a GNN-only ablation. We further validate scalability on the ForgeHLS dataset. Index Terms—high-level synthesis, QoR prediction, graph neural networks, differential learning, large language models
I. I NTRODUCTION HLS compiles C/C++ into RTL and is widely used to build FPGA accelerators. In practice, designers explore pragma choices such as pipelining and unrolling to trade off performance and resource usage, but each design point can require a costly HLS run. QoR prediction estimates post-HLS design metrics to accelerate design space exploration (DSE); typical targets span resource utilization metrics such as DSP, FF, and LUT, and timing metrics such as the critical-path delay (CP). GNN-based predictors over IR graphs are effective [1], [2], but they must handle large inter-kernel target variance and may miss pragmaaware semantic cues that remain explicit in source code. This paper introduces D IFF HLS, a differential QoR predictor that learns from kernel–design pairs. We treat the original kernel as a baseline and a pragma-inserted implementation as a design variant, and train the model to predict both the kernel baseline and the design-induced delta. We further
incorporate frozen LLM code embeddings to complement IR graph signals. The key design rationale behind D IFF HLS rests on two observations: 1) Predicting changes is more precise than predicting absolutes. In pragma-driven datasets, each kernel induces a cluster of design points that share a large kernel-dependent baseline but differ due to pragma choices. Directly regressing the design-level target forces the model to simultaneously explain the large baseline differences across kernels and the smaller, pragma-driven variations within each kernel. By decomposing the prediction into a kernel baseline and a designinduced delta, we reduce target variance within each cluster, let the model focus on pragma effects, and gain a natural diagnostic: errors can be attributed to either baseline estimation or delta modeling. 2) Graphs and code carry different but complementary information. In HLS optimization, pragma directives and codelevel patterns such as loop structure and memory-access style strongly influence resource usage. IR graphs capture control and data dependencies across the program, while source code retains explicit pragma annotations and local patterns that may be weakened during IR lowering. We therefore combine graph representations with pretrained LLM code embeddings: graphs model structural dependencies, and code embeddings provide pragma-aware semantic context. Our contributions are: We propose D IFF HLS, a differential learning formulation for HLS QoR prediction that models each pragma-inserted design as a kernel baseline plus a learned delta. • We design a hybrid predictor that fuses CDFG-based GNN representations with pretrained LLM code embeddings to capture both IR-level structure and pragma-aware code semantics. • We evaluate on PolyBench with targeted ablations that quantify the contributions of differential learning, code
•
embeddings, and component-level error sources, and Algorithm 1 Training of D IFF HLS Input: paired samples {(Gk , Gd , yk , yd )}, code tokens t further validate scalability on the larger ForgeHLS dataset. for each minibatch do II. R ELATED W ORK ∆ ← yd − yk hk ← GNNEnck (Gk ); hd ← GNNEncd (Gd ) A. HLS QoR Prediction if Code Emb. then QoR prediction for HLS has been widely studied. Common hc ← CodeEnc(t) benchmarks include PolyBench/C [3], DB4HLS [4], and h∆ ← [hk ; hd ; hc ] HLSyn [5]. Larger-scale datasets such as ForgeHLS [6] and else HLS4ML [7] expand diversity. Early work leverages handh∆ ← [hk ; hd ] crafted features from HLS reports or compiler IR with classical end if regressors [8]–[10]. Recent approaches represent programs as b ← f∆ (h∆ ) ŷk ← fk (hk ); ∆ IR graphs and apply GNNs; representative examples include b ∆) + ℓ(ŷk + ∆, b yd ) L ← ℓ(ŷk , yk ) + ℓ(∆, CDFG-based prediction [1], hierarchical and heterogeneous Update parameters using ∇L extensions [2], [11]–[13], post-route estimation [14], and end for GNN-based area and timing prediction [15]. Hierarchy-aware encoders target long-range dependencies from nested loops [16], [17]; related modeling has been studied for pre-HLS power the original kernel code without pragma directives, while yd prediction [18]. Cross-modal approaches fuse source-code represents the corresponding metric of the pragma-inserted tokens with IR graphs via transformers and GNNs [19]. design code. Mixture-of-experts routing has been proposed for specialized prediction [20]. These methods typically regress absolute QoR B. Graph Encoding targets per design, making it hard to capture the subtle effects We build IR graphs Gk and Gd for the kernel and design, of individual pragmas accurately. D IFF HLS addresses this by decomposing prediction into a kernel baseline plus a pragma- respectively, following the CDFG (control-data flow graph) representation of Wu et al. [1]. The graphs are extracted from induced delta. the HLS toolchain’s ADB intermediate representation, which B. LLM Code Embeddings captures both control flow and data dependencies in a unified Pretrained code models provide useful representations for graph structure. We encode them with two parameter-disjoint graph encoders downstream code understanding; encoder-style models such as CodeBERT [21] and GraphCodeBERT [22] learn semantic for the kernel and design branches, with the same architecture embeddings that can complement graph-structured IR features. but independent parameters: Since many strong code LMs are decoder-only, recent work hk = GNNEnck (Gk ), hd = GNNEncd (Gd ). (2) studies how to extract encoder-style representations from them [23]; other work explores using LM decoding directly Here hk , hd ∈ RD . As illustrated in Fig. 1, each encoder for regression [24]. Code embeddings can complement IR stacks N GNN layers, where each layer consists of a graph graphs by retaining pragma directives and local semantic cues convolution, followed by GraphNorm and a ReLU nonlinearity. that may be weakened during IR lowering; this motivates A sum pooling readout aggregates node representations into a lightweight fusion of pretrained code representations with graph-level embedding. graph encoders [21], [22]. D IFF HLS incorporates a lightweight, We follow Wu et al. [1] for node and edge attribute frozen LLM code embedding as an auxiliary design feature, construction. avoiding expensive LLM fine-tuning while improving sensitivity to pragma-driven changes. C. LLM Code Embedding III. M ETHOD IR graphs capture compiler-level structure but may miss source-level patterns relevant to optimization. We therefore A. Problem Formulation augment each design d with a frozen code encoder g(·) from a For each kernel k and one of its pragma-inserted designs pretrained code LLM. Let t denote the token sequence obtained d, we predict a QoR target yd ∈ R, where the target is one by concatenating the design source files. We compute a fixedof DSP, FF, LUT, or CP. We treat each design as a paired length representation by extracting the last token’s hidden state: sample (k, d) and decompose the design-level target into a kernel baseline yk and a design-induced delta: z = g(t) . (3) last
∆ = yd − yk .
(1)
This formulation reduces inter-kernel scale variance and centers learning on pragma effects. Both yk and yd are obtained from post-HLS synthesis reports: yk represents the QoR metric of
Since the code embedding dimension L may differ from the GNN hidden size D, we use a lightweight adapter a : RL → RD : hc = a(z). (4)
Graph Embeddings GNNLayer
Kernel Code
Pragma
Code Embeddings Design Code Tokens
GNNConv Kernel Graph
Design Graph
ReLU
Head
Transformer Node Emb.
GraphNorm GNNLayer × N
GNNLayer × N
Sum Pooling
Sum Pooling
Last Token Pooling
Projection Graph Emb.
Code Emb.
Linear GELU Dropout
Kernel Head
Delta Head
Linear
Fig. 1. Overview of D IFF HLS. We encode kernel and design IR graphs, inject LLM code embeddings into the delta pathway, and predict the kernel baseline and delta; finally the design prediction is obtained by composition.
void vector_add(int a[8], int b[8], int c[8]) { for (int i = 0; i < 8; i++) { c[i] = a[i] + b[i]; } } Pragma insertion void vector_add(int a[8], int b[8], int c[8]) { #pragma HLS ARRAY_PARTITION variable=a factor=2 #pragma HLS ARRAY_PARTITION variable=b factor=2 #pragma HLS ARRAY_PARTITION variable=c factor=8 for (int i = 0; i < 8; i++) { #pragma HLS PIPELINE OFF #pragma HLS UNROLL factor=2 c[i] = a[i] + b[i]; } }
The detailed architecture is illustrated in Fig. 1. The kernel head estimates the kernel baseline solely from the kernel embedding: ŷk = fk (hk ),
where fk : RD → R is a small MLP regressor. Concretely, both heads use the same MLP template, differing only in input dimension. The delta head targets the design-induced change and therefore conditions on both the kernel and design embeddings, optionally augmented with the code embedding hc , as shown in Algorithm 1: ( [hk ; hd ], w/o code emb. h∆ = (6) [hk ; hd ; hc ], w/ code emb. b = f∆ (h∆ ), ∆
Fig. 2. Example kernel code (top) and design code (bottom) illustrating HLS pragma insertion.
(5)
(7)
where f∆ : R2D → R or R3D → R is another MLP. We then compose the final design prediction as:
Thus hc ∈ RD . We inject LLM code embeddings only into the b ŷd = ŷk + ∆. (8) delta pathway via the delta head, to preserve the interpretation This two-head design separates the kernel-dependent baseline of yk as a GNN-only kernel baseline. For each design d, we use its pragma-inserted C/C++ source from pragma-driven changes. Code embeddings enter only files as the code input. We concatenate the design files into a through the delta pathway, so ŷk remains a graph-only kernel single token sequence and feed it to the frozen code encoder. We baseline. use only design code because the kernel baseline yk is predicted E. Training Objective from the kernel IR graph alone, while code embeddings serve We supervise kernel, delta, and design outputs with a as an auxiliary signal for the design delta. Figure 2 shows an SmoothL1 loss ℓ(·, ·): example kernel and its corresponding pragma-inserted design. b ∆) + ℓ(ŷk + ∆, b yd ), L = ℓ(ŷk , yk ) + ℓ(∆, (9) D. Differential Heads where all three terms are equally weighted. The first two terms To implement the decomposition in Eq. (1), we use two directly supervise the kernel baseline prediction and the delta prediction heads that are trained jointly but serve distinct roles. prediction, respectively. The third term enforces consistency
TABLE I TARGET VALUE RANGES ( KERNEL , DELTA , DESIGN ).
10
[min, max]
Kernel
Delta
Design
DSP FF LUT CP
[3, 29] [495, 3994] [1214, 5192] [5.31, 7.21]
[−7, 75] [−174, 25414] [151, 35865] [−0.10, 0.78]
[5, 94] [1296, 28967] [1842, 39464] [5.55, 7.75]
Avg. MAPE (%)
6
Backbone
Method
DSP
FF
LUT
CP
0
PNA
GNN [1] HGNN [2] D IFF HLS
14.71 7.84 3.63
26.47 9.65 8.18
22.86 10.55 5.52
8.87 1.34
GraphSAGE
GNN [1] HGNN [2] D IFF HLS
17.01 6.94 3.31
39.11 9.99 7.34
28.09 9.86 5.12
8.25 1.06
GCN
GNN [1] HGNN [2] D IFF HLS
25.30 7.45 8.10
38.34 11.24 9.31
28.64 11.27 5.46
8.79 1.37
GAT
GNN [1] HGNN [2] D IFF HLS
28.66 8.18 6.41
54.73 11.57 9.63
46.19 12.34 6.11
10.32 1.53
IV. E XPERIMENTAL S ETUP A. Metrics We report mean absolute error, denoted MAE; mean absolute percentage error, denoted MAPE and reported in percent; and coefficient of determination, denoted R2 . Lower is better for MAE and MAPE, and higher is better for R2 . For a set of N designs with targets {yi } and predictions {ŷi }, MAE, MAPE, and R2 are: N
MAPE =
N0
,
yi i: yi ̸=0 PN (yi − ŷi )2 R2 = 1 − Pi=1 . N 2 i=1 (yi − ȳ)
6.3
5.9
5.2 4.7
PNA
GraphSAGE
GCN
GAT
Fig. 3. Average MAPE (%) across DSP, FF, LUT, and CP for the three method variants under four GNN backbones.
B. Dataset We evaluate on 27 PolyBench/C kernels [3] and 10,108 design points derived from the ForgeHLS PolyBench subset [6] via pragma insertion. We predict design-level QoR targets yd ∈ {DSP, FF, LUT, CP}, whose value ranges are listed in Table I. We use a design-level random split with an 8:1:1 train/validation/test ratio over paired kernel–design samples. We train separate models for each of DSP, FF, LUT, and CP. C. Implementation Details
b and the ground-truth between the composed prediction ŷk + ∆ design target yd . This three-term objective serves two purposes. First, it ensures that both the kernel head and delta head learn meaningful predictions individually. Second, it prevents error accumulation when composing the two predictions. Without b could accumulate the consistency term, errors in ŷk and ∆ and produce larger errors in the final design prediction. The b yd ) penalizes such misalignment consistency term ℓ(ŷk + ∆, and encourages the two heads to cooperate.
1 X |yi − ŷi |, N i=1 100 X yi − ŷi
6.1
6.0 5.1
4.2
2
We report MAPE in %. Best results are highlighted in bold.
6.1
8.3 7.2 7.1
4
TABLE II Q O R PREDICTION ACCURACY UNDER DIFFERENT GNN BACKBONES .
MAE =
w/o diff w/o code emb. DiffHLS
8
(10)
PN Here N0 = {i : yi ̸= 0} , ȳ = N1 i=1 yi , and MAPE excludes samples with zero targets to avoid division-by-zero.
Unless otherwise stated, we train with Adam optimizer, learning rate 5 × 10−4 , batch size 16, hidden size 128, two GNN layers, and dropout 0.02. We use ReduceLROnPlateau with patience 15 and factor 0.8, and select the best checkpoint based on validation loss. All experiments are run on a single NVIDIA RTX 4090 GPU with 24 GB memory. V. E XPERIMENTAL R ESULTS A. Main Results Table II summarizes design-level QoR prediction MAPE under four GNN backbones. D IFF HLS achieves strong performance across all backbones, with GraphSAGE yielding the best results. Compared to the GNN baseline [1], D IFF HLS reduces MAPE substantially on every target under all backbones. For example, under GraphSAGE, DSP MAPE drops from 17.01% to 3.31% and LUT MAPE drops from 28.09% to 5.12%. B. Differential vs. Direct Prediction Table III compares D IFF HLS with a direct prediction variant (D IFF HLS w/o diff) that disables the differential formulation and directly predicts design targets. Disabling differential learning consistently increases MAPE on FF and LUT across all backbones. The degradation is particularly large on LUT under GAT, where MAPE increases from 6.11% to 13.39%. This validates our core hypothesis: by decomposing yd = yk + ∆, the model only needs to learn the pragma-induced change ∆ rather than the full target value, which varies by orders of magnitude across different kernels. The differential formulation therefore simplifies the learning problem considerably. Fig. 3 visualizes the average MAPE of the three variants across all backbones.
TABLE III A BLATION STUDY: DIFFERENTIAL VS . DIRECT PREDICTION AND EFFECT OF CODE EMBEDDINGS . MAPE ↓
DSP MAE ↓
R2 ↑
MAPE ↓
FF MAE ↓
R2 ↑
MAPE ↓
LUT MAE ↓
R2 ↑
MAPE ↓
CP MAE ↓
R2 ↑
PNA
D IFF HLS (w/o diff) D IFF HLS (w/o code emb.) D IFF HLS
3.86 4.18 3.63
1.16 1.39 1.21
0.980 0.976 0.899
10.75 8.51 8.18
725.94 677.34 629.84
0.972 0.963 0.973
8.50 6.30 5.52
891.66 641.35 627.03
0.969 0.967 0.989
1.22 1.22 1.34
0.090 0.090 0.099
0.711 0.735 0.710
GraphSAGE
D IFF HLS (w/o diff) D IFF HLS (w/o code emb.) D IFF HLS
4.19 5.11 3.31
1.25 1.81 1.28
0.971 0.945 0.964
10.60 8.64 7.34
680.01 698.86 647.85
0.926 0.974 0.897
7.76 5.78 5.12
707.05 576.78 510.83
0.935 0.991 0.991
1.31 1.24 1.06
0.096 0.092 0.078
0.679 0.721 0.847
GCN
D IFF HLS (w/o diff) D IFF HLS (w/o code emb.) D IFF HLS
8.91 10.08 8.10
2.78 3.34 2.84
0.937 0.915 0.950
10.71 10.37 9.31
897.59 996.00 1001.49
0.935 0.678 0.413
7.83 6.68 5.46
776.59 751.33 621.47
0.986 0.981 0.985
1.22 1.27 1.37
0.090 0.094 0.101
0.718 0.732 0.664
GAT
D IFF HLS (w/o diff) D IFF HLS (w/o code emb.) D IFF HLS
7.67 8.07 6.41
2.57 2.60 2.18
0.938 0.952 0.958
10.88 9.98 9.63
820.99 763.87 905.99
0.947 0.967 0.863
13.39 5.88 6.11
1186.50 562.37 715.22
0.969 0.991 0.982
1.32 1.28 1.53
0.098 0.095 0.113
0.294 0.700 0.628
Backbone
Method
D IFF HLS (w/o diff) disables differential learning and directly predicts design targets without the kernel–design delta decomposition. D IFF HLS (w/o code emb.) disables code embeddings. TABLE IV A BLATION STUDY: PREDICTION HEAD ACCURACY. Head
Kernel ŷk
b Delta ∆
Backbone
Code Emb.
MAPE
DSP MAE
R2
MAPE
FF MAE
R2
MAPE
LUT MAE
R2
MAPE
CP MAE
R2
PNA
No Yes
0.43 0.84
0.08 0.12
1.000 0.999
0.74 0.66
15.53 15.08
0.999 1.000
0.55 0.38
13.36 8.81
1.000 1.000
0.06 0.12
0.004 0.008
0.993 0.933
GraphSAGE
No Yes
0.89 0.51
0.14 0.09
0.999 1.000
0.94 0.86
20.13 20.48
1.000 1.000
0.58 0.61
13.54 13.83
1.000 1.000
0.11 0.07
0.008 0.005
0.955 0.998
GCN
No Yes
1.20 0.86
0.15 0.14
0.999 0.999
0.67 0.74
13.44 16.36
1.000 1.000
0.82 0.38
19.48 9.05
0.999 1.000
0.06 0.24
0.004 0.017
0.996 0.379
GAT
No Yes
0.79 1.63
0.12 0.25
0.999 0.997
2.09 1.01
51.36 21.56
0.994 0.999
0.91 1.16
22.56 30.21
0.999 0.998
0.10 0.12
0.007 0.008
0.992 0.971
PNA
No Yes
13.55 11.37
1.38 1.23
0.971 0.872
58.52 66.76
676.20 631.17
0.961 0.972
13.71 12.07
641.63 627.14
0.965 0.988
35.80 38.13
0.090 0.099
0.709 0.654
GraphSAGE
No Yes
18.59 11.92
1.81 1.28
0.931 0.956
53.23 46.15
695.03 650.10
0.973 0.891
11.83 10.63
576.72 511.54
0.990 0.991
36.61 31.33
0.092 0.078
0.706 0.740
GCN
No Yes
38.82 26.43
3.37 2.86
0.895 0.938
59.39 32.41
995.09 1002.12
0.661 0.384
13.60 11.72
754.88 622.58
0.980 0.984
36.13 37.80
0.093 0.102
0.675 0.638
GAT
No Yes
29.27 22.45
2.59 2.17
0.939 0.948
125.26 59.00
754.27 906.03
0.966 0.857
13.14 12.63
563.01 713.02
0.991 0.981
33.77 39.63
0.094 0.112
0.657 0.585
C. Effect of Code Embeddings Table III also compares D IFF HLS with a GNN-only variant (D IFF HLS w/o code emb.) that removes code embeddings from the delta pathway. Adding code embeddings consistently decreases MAPE across DSP, FF, and LUT under all backbones. The largest impact is on DSP, where MAPE decreases from 5.11% to 3.31% under GraphSAGE. This suggests that LLM code embeddings capture pragma annotations and loop structures that may be lost during IR generation, providing complementary cues to the graph-based features.
from 13.55% to 11.37% under PNA and from 18.59% to 11.92% under GraphSAGE. These results confirm that the kernel baseline is an easy subproblem, while the delta head handles the harder pragma-driven variation. E. GNN Backbone Across all experiments, GraphSAGE consistently yields the best results, outperforming PNA, GCN, and GAT on all QoR targets. The differential formulation benefits all four backbones substantially, and code embeddings provide further gains under GraphSAGE. Overall, D IFF HLS is robust to the choice of GNN backbone.
D. Prediction Head Accuracy
F. Code LLM Backbone
Table IV reports accuracy for the kernel head and delta head separately. The kernel head achieves MAPE below 2% and R2 above 0.99 on DSP, FF, and LUT, confirming that kernel baselines are well-captured by IR graph structure alone. The delta head is more challenging due to the wide range of pragma-induced QoR changes, but still maintains reasonable R2 . Adding code embeddings reduces delta MAPE on DSP
Table V compares two code LLM backbones for embedding extraction under the same training protocol. Under PNA and GraphSAGE, Qwen2.5-Coder-1.5B achieves lower MAPE on DSP and LUT, while Llama-3.2-1B sometimes yields better MAE and R2 on FF. Under GCN, Llama-3.2-1B achieves lower MAPE on DSP, FF, and CP, while Qwen2.5-Coder-1.5B retains an advantage on LUT. Under GAT, Llama-3.2-1B outperforms
TABLE V A BLATION STUDY: CODE LLM BACKBONE . Backbone
Model
MAPE ↓
DSP MAE ↓
R2 ↑
MAPE ↓
FF MAE ↓
R2 ↑
MAPE ↓
LUT MAE ↓
R2 ↑
MAPE ↓
CP MAE ↓
R2 ↑
PNA
Qwen2.5-Coder-1.5B Llama-3.2-1B
3.63 3.96
1.21 1.40
0.899 0.979
8.18 8.09
629.84 524.57
0.973 0.967
5.52 5.60
627.03 457.07
0.989 0.984
1.34 1.13
0.099 0.084
0.710 0.776
GraphSAGE
Qwen2.5-Coder-1.5B Llama-3.2-1B
3.31 4.52
1.28 1.71
0.964 0.926
7.34 8.14
647.85 528.27
0.897 0.963
5.12 5.56
510.83 453.39
0.991 0.989
1.06 1.17
0.078 0.086
0.847 0.738
GCN
Qwen2.5-Coder-1.5B Llama-3.2-1B
8.10 8.08
2.84 2.73
0.950 0.947
9.31 8.93
1001.49 817.44
0.413 0.947
5.46 6.50
621.47 717.54
0.985 0.988
1.37 1.13
0.101 0.083
0.664 0.774
GAT
Qwen2.5-Coder-1.5B Llama-3.2-1B
6.41 5.91
2.18 2.09
0.958 0.957
9.63 9.31
905.99 685.07
0.863 0.958
6.11 6.06
715.22 602.63
0.982 0.991
1.53 1.20
0.113 0.088
0.628 0.628
TABLE VI F ORGE HLS LARGE - SCALE EVALUATION . Backbone
DSP
FF
LUT
CP
PNA GraphSAGE GCN GAT
5.70 5.69 9.11 10.75
25.75 17.30 25.18 20.16
10.05 7.77 11.94 9.27
4.21 4.93 5.83 4.41
We report design-level MAPE in %.
Qwen2.5-Coder-1.5B on most metrics. These results suggest that the optimal code backbone may depend on the GNN architecture and target metric. Overall, we use Qwen2.5-Coder1.5B as the default code embedding backbone. G. ForgeHLS Large-Scale Evaluation To further validate scalability, we evaluate D IFF HLS on the ForgeHLS large-scale dataset [6]. All runs use the full D IFF HLS configuration with differential heads and code embeddings enabled. Table VI reports design-level MAPE for four GNN backbones. GraphSAGE delivers the strongest overall performance, achieving the lowest MAPE on resource targets. PNA achieves the lowest CP MAPE. The ranking among backbones is consistent with the PolyBench results, confirming that D IFF HLS generalizes to a substantially larger and more diverse design space. VI. C ONCLUSION We presented D IFF HLS, a differential learning framework for HLS QoR prediction that learns from kernel–design pairs by decomposing design-level targets into a kernel baseline and a design-induced delta, and augments graph representations with LLM code embeddings. On PolyBench, D IFF HLS improves design-level prediction accuracy over baselines under four GNN backbones across all four QoR targets. Our ablations show that differential learning provides the largest gain, and code embeddings further improve performance. We also validate scalability on the larger ForgeHLS dataset. R EFERENCES [1] N. Wu, H. Yang, Y. Xie, P. Li, and C. Hao, “High-level synthesis performance prediction using graph neural networks: Benchmarking, modeling, and advancing,” in Proceedings of the ACM/IEEE Design Automation Conference (DAC), 2022. [2] M. Gao, J. Zhao, Z. Lin, and M. Guo, “Hierarchical source-to-postroute qor prediction in high-level synthesis with gnns,” in 2024 Design, Automation & Test in Europe Conference & Exhibition (DATE). IEEE, 2024, pp. 1–6.
[3] L.-N. Pouchet and T. Yuki, “Polybench/c 4.2,” https://sourceforge.net/ projects/polybench/files/polybench-c-4.2/, 2016, accessed: 2025-11-20. [4] L. Ferretti, J. Kwon, G. Ansaloni, G. Di Guglielmo, L. Carloni, and L. Pozzi, “Db4hls: A database of high-level synthesis design space explorations,” IEEE Embedded Systems Letters, vol. 13, no. 4, pp. 194– 197, 2021. [5] Y. Bai, A. Sohrabizadeh, Z. Qin, Z. Hu, Y. Sun, and J. Cong, “Towards a comprehensive benchmark for high-level synthesis targeted to FPGAs,” Advances in Neural Information Processing Systems, vol. 36, pp. 45 288– 45 299, 2023. [6] Z. Peng, Z. Li, M. Gao, Q. Xu, C. Zhang, and J. Zhao, “ForgeHLS: A Large-Scale, Open-Source Dataset for High-Level Synthesis,” Aug. 2025, arXiv:2507.03255 [cs]. [Online]. Available: http://arxiv.org/abs/2507.03255 [7] B. Hawks, J. Weitz, D. Demler, K. Tame-Narvaez, D. Plotnikov, M. M. Rahimifar, H. E. Rahali, A. C. Therrien, D. Sproule, E. E. Khoda et al., “wa-hls4ml: A benchmark and surrogate models for hls4ml resource and latency estimation,” arXiv preprint arXiv:2511.05615, 2025. [8] G. Zhong, A. Prakash, S. Wang, Y. Liang, T. Mitra, and S. Niar, “Design space exploration of fpga-based accelerators with multi-level parallelism,” in Design, Automation & Test in Europe Conference & Exhibition (DATE), 2017. IEEE, 2017, pp. 1141–1146. [9] S. Dai, Y. Zhou, H. Zhang, E. Ustun, E. F. Young, and Z. Zhang, “Fast and accurate estimation of quality of results in high-level synthesis with machine learning,” in 2018 IEEE 26th Annual International Symposium on Field-Programmable Custom Computing Machines (FCCM). IEEE, 2018, pp. 129–132. [10] J. Zhao, L. Feng, S. Sinha, W. Zhang, Y. Liang, and B. He, “Comba: A comprehensive model-based analysis framework for high level synthesis of real applications,” in 2017 IEEE/ACM International Conference on Computer-Aided Design (ICCAD). IEEE, 2017, pp. 430–437. [11] A. Sohrabizadeh, Y. Bai, Y. Sun, and J. Cong, “Automated accelerator optimization aided by graph neural networks,” in Proceedings of the 59th ACM/IEEE Design Automation Conference, 2022, pp. 55–60. [12] Z. Lin, Z. Yuan, J. Zhao, W. Zhang, H. Wang, and Y. Tian, “Powergear: Early-stage power estimation in fpga hls via heterogeneous edge-centric gnns,” in Procs. of Design, Automation and Test in Europe Conference and Exhibition (DATE), 2022. [13] Z. Lin, J. Zhao, S. Sinha, and W. Zhang, “Hl-pow: A learning-based power modeling framework for high-level synthesis,” in 2020 25th Asia and South Pacific Design Automation Conference (ASP-DAC), 2020, pp. 574–580. [14] P. Goswami and D. Bhatia, “Predicting post-route quality of results estimates for HLS designs using machine learning,” in 2022 23rd International Symposium on Quality Electronic Design (ISQED). IEEE, 2022, pp. 45–50. [15] M. U. Jamal, Z. Li, M. T. Lazarescu, and L. Lavagno, “A graph neural network model for fast and accurate quality of result estimation for high-level synthesis,” IEEE Access, 2023. [16] A. Sohrabizadeh, Y. Bai, Y. Sun, and J. Cong, “Robust GNN-based representation learning for HLS,” in 2023 IEEE/ACM International Conference on Computer Aided Design (ICCAD). IEEE, 2023, pp. 1–9. [17] E. Murphy and L. Josipović, “Balor: Hls source code evaluator based on custom graphs and hierarchical gnns,” in Proceedings of the 43rd IEEE/ACM International Conference on Computer-Aided Design, 2024, pp. 1–9.
[18] Z. Lin, Z. Peng, M. Gao, J. Zhao, and Z. Lin, “Hippo: A hierarchypreserving and noise-tolerant pre-hls power modeling framework for fpga,” in Proceedings of the IEEE/ACM International Conference on Computer-Aided Design (ICCAD), 2025. [19] Z. Qin, Y. Bai, A. Sohrabizadeh, Z. Ding, Z. Hu, Y. Sun, and J. Cong, “Cross-modality program representation learning for electronic design automation with high-level synthesis,” arXiv preprint arXiv:2406.09606, 2024. [20] W. Li, D. Wang, Z. Ding, A. Sohrabizadeh, Z. Qin, J. Cong, and Y. Sun, “Hierarchical mixture of experts: Generalizable learning for highlevel synthesis,” in Proceedings of the AAAI Conference on Artificial Intelligence, vol. 39, no. 17, 2025, pp. 18 476–18 484. [21] Z. Feng, D. Guo, D. Tang, N. Duan, X. Feng, M. Gong, L. Shou, B. Qin, T. Liu, and D. Jiang, “Codebert: A pre-trained model for programming and natural languages,” arXiv preprint arXiv:2002.08155, 2020. [22] D. Guo, S. Ren, S. Lu, Z. Feng, D. Tang, S. Liu, L. Zhou, N. Duan, A. Svyatkovskiy, S. Fu et al., “Graphcodebert: Pre-training code representations with data flow,” arXiv preprint arXiv:2009.08366, 2020. [23] P. Suganthan, F. Moiseev, L. Yan et al., “Adapting decoder-based language models for diverse encoder downstream tasks,” 2025, arXiv preprint. [24] X. Song and D. Bahri, “Decoding-based regression,” 2025, arXiv preprint.