NLP-based Message Prioritisation for Opportunistic Routing: A Case Study of Nepal Earthquake Emergency Communication

arXiv:2610.37933v1 [cs.NI] 27 Oct 2026

NLP-based Message Prioritisation for Opportunistic Routing: A Case Study of Nepal Earthquake Emergency Communication Yihang Cao

Milena Radenkovic

School of Computer Science University of Nottingham Nottingham, United Kingdom alyyc111@nottingham.ac.uk

School of Computer Science University of Nottingham Nottingham, United Kingdom milena.radenkovic@nottingham.ac.uk

Abstract—This dissertation investigates whether Natural Language Processing (NLP) based semantic message prioritisation can improve the treatment of urgent disaster messages in opportunistic emergency communication, using the 2015 Nepal earthquake as a case study. TF IDF with Logistic Regression and BERT are used to classify social media messages into nine humanitarian categories, which are mapped to four routing priorities from P1 to P4. These priorities are integrated into the ONE simulator within a controlled Kathmandu earthquake scenario using a modified Binary Spray and Wait router. BERT achieves higher priority accuracy and P1 recall than Logistic Regression, while reducing under prioritisation and severe priority errors. However, improved NLP performance does not produce uniform network gains. P1 delivery probability remains high and similar across routing conditions, and no consistent latency reduction is observed. The clearest difference appears under buffer pressure, where the BERT condition shows fewer observed ground truth P1 buffer losses than Logistic Regression. The results indicate that semantic prioritisation is most valuable when limited transmission and storage resources are contested, although it cannot overcome mobility and contact constraints in opportunistic networks. Keywords—Natural Language Processing, Opportunistic Routing, Message Prioritisation, Disaster Communication, BERT, Logistic Regression

I. I NTRODUCTION Natural disasters can disrupt conventional communication infrastructure when reliable information exchange is most critical [1], [2]. Opportunistic networking provides an alternative through a store-carry-forward mechanism, enabling mobile devices to exchange messages without continuous endto-end connectivity [3], [4]. However, limited transmission opportunities and buffer capacity make it difficult to ensure that urgent disaster messages receive appropriate treatment [5]–[7]. Existing opportunistic routing approaches primarily consider network conditions and resource availability, while message-level humanitarian urgency may remain unaddressed. Natural language processing (NLP) offers a means of identifying humanitarian information from disaster-related text and

translating message semantics into routing priorities [8]–[10]. However, classification errors may cause genuinely urgent messages to receive insufficient priority during transmission scheduling and buffer management [11], [12]. Although message classification and priority-aware routing have been studied separately, their end-to-end interaction requires further investigation. This paper investigates whether NLP-based semantic message prioritisation can improve the treatment of urgent messages in opportunistic emergency communication. Using social media messages from the 2015 Nepal earthquake [5], [13], we compare TF–IDF with Logistic Regression and BERT [14] for humanitarian message classification. The predicted categories are mapped to four routing priority levels and integrated into a modified Binary Spray and Wait protocol [15] in the ONE simulator [16], using a controlled Kathmandu earthquake scenario. The main contribution of this study is an end-to-end, crosslayer evaluation of NLP-based semantic message prioritisation in opportunistic disaster communication. Rather than evaluating humanitarian text classification in isolation, the study integrates predicted message priorities into a modified Binary Spray and Wait protocol in the ONE simulator. The evaluation compares standard routing, oracle priority routing, and routing based on priorities predicted by TF–IDF with Logistic Regression and BERT, examining classification errors and their downstream effects on the delivery, latency, and buffer loss of urgent messages. Together, these analyses investigate whether improvements in semantic priority prediction translate into better treatment of urgent traffic under constrained network conditions. II. R ELATED W ORK A. Opportunistic Routing and Resource Management Delay-tolerant and opportunistic networks support communication under intermittent connectivity through a storecarry-forward mechanism [17]–[20]. Current routing protocols consider trade-off between delivery opportunities and resource usage. Epidemic routing uses high replication to offer more

delivery opportunities at the cost of high bandwidth and buffer usage [21]. PRoPHET uses encounter history to forward messages [22], Spray and Wait uses limited replication to minimize communication overhead [15]. Contextual routing has also been studied in disaster settings [23]. Fixed replication strategy might not be sufficient to address varying congestion and resource conditions. Radenkovic and Grundy have studied congestion-aware forwarding based on buffer, delay and contact information to avoid congested areas of the network. [24]. The CafRep proposed in further studies adaptive congestion control and replication by adapting forwarding strategy based on local congestion and resource variations. [25]. This shows that forwarding strategy should be adapted to the current network condition instead of using fixed replication strategy. Energy-aware opportunistic charging and energy distribution has been studied to consider resource limitation from the energy management point of view [26], [27]. Huynh and Radenkovic have proposed cross-layer emergency communication scheme with adaptive storage, packet scheduling and utility-based forwarding [28]. Collaborative cognitive content dissemination and retrieval has also been studied [29]. And multi-agent reinforcement learning for edge caching has been considered in subsequent work [30]. Predictive topology-aware resource provisioning is another way to adapt resource to current demandallocation to changing demand [31]. Related studies on energy-aware opportunistic charging and energy distribution address resource constraints from an energymanagement perspective [32], [33]. These works show that current network status, contextual information and predictive information can be used to make forwarding and resource allocation decisions. Priority-aware routing can tackle a different problem by treating different messages differently. Current solutions can be used to schedule transmission and buffer messages based on priority [11], [12], [34], [35]. However, these solutions do not necessarily indicate how to derive humanitarian priority from unstructured disaster text. B. NLP for Humanitarian Message Classification NLP can be used to classify social media messages related to disasters into humanitarian types to help discover useful information during a disaster [5], [36]. Most of the current methods use statistical text features (e.g., TF-IDF) and machine learning classifiers (e.g., Logistic Regression and SVM) [8], [37], [38]. While these methods are computationally lightweight, they may not be able to capture contextual information due to their use of lexical features [5], [36], [39]. Transformer-based humanitarian classification models like BERT can offer contextual message representations and have been utilized in crisis classification [14], [40]. However, humanitarian classification is difficult to perform due to short, casual, and contextual nature of messages as well as imbalanced category distribution [41], [42]. Accuracy might hide the undesirable results for less frequent yet operationally significant categories. Class-aware evaluation measures, recall

of urgent category, and error analysis in a certain direction can give more insights into the classification reliability [43]. This is especially relevant when the classification results are used for other purposes than just categorizing the messages. In case the predicted humanitarian category impacts the priority of message routing, the classification error can affect the decision on message forwarding, storage, or even discarding. Thus, the operational impact of the classifier depends on its classification results as well as the impact of the error on the further network decision. C. Semantic Prioritisation and Research Gap Prioritization based on content has been studied in delaytolerant networks [44], [45], and priority-aware routing has been researched in terms of different treatment of messages during routing and buffering [11], [12]. Congestion-aware routing, cross-layer emergency communication, and adaptive resource management have been shown to allow the network to make decisions depending on the current resource state [24], [25], [28]. NLP for humanitarian messages can provide another way of obtaining semantic priority of messages [5], [8]. However, this raises a question of how the automatically predicted humanitarian priority impacts downstream opportunistic routing. In contrast to the case when the priority is predefined, routing based on NLP can have uncertainty in the decision on resource allocation due to classification. In case the priority of an urgent message is not high enough, it can be treated less favourably, and if the priority is predicted too high, it can be allocated resources that are needed for higher priority traffic. Moreover, better classification does not always lead to better delivery and lower latency as mobility, availability of contacts, and buffer size can also impact the results. The proposed work fills this gap by combining NLP humanitarian message classification with priority-aware opportunistic routing and studying the impact of predicted semantic priority on delivery, latency, and buffer protection of urgent messages with respect to conventional routing and ground truth priority. The cross-layer approach allows to study the impact of classification reliability on emergency communication. III. M ETHODOLOGY A. Nepal Earthquake Study Design This study investigates whether NLP-derived semantic priorities can improve the treatment of urgent disaster messages in opportunistic networks, using the 2015 Nepal earthquake as a case study. Humanitarian messages are classified into nine categories and mapped to four routing priority levels, which are incorporated into a modified Binary Spray and Wait protocol in The ONE simulator. To evaluate the effects of semantic prioritisation, four routing conditions are compared: standard Binary Spray and Wait without semantic priorities, oracle priority routing using ground-truth priorities, Logistic Regression priority routing, and BERT priority routing. All conditions use the same message schedule, Kathmandu road map, node configuration, and

mobility seeds to support a controlled comparison of network performance. Ground-truth priorities are retained separately from predicted priorities to evaluate how classification errors affect the treatment of genuinely urgent messages.

B. Nepal Earthquake NLP–Routing Architecture Design The proposed architecture integrates NLP-based humanitarian message classification with priority-aware opportunistic routing in The ONE simulator. As illustrated in Fig. 1, the framework comprises four stages: data and NLP processing, NLP-to-simulation integration, opportunistic routing simulation, evaluation and analysis.

TABLE I N EPAL E ARTHQUAKE DATASET AND S PLIT S UMMARY. Dataset Characteristic

Value

Cleaned messages Training messages Test messages Training proportion Test proportion Humanitarian categories Routing priority levels Random seed

3,003 2,402 601 80% 20% 9 4 42

The distribution of humanitarian message categories is imbalanced, as shown in Fig. 2. Donation and volunteering messages form the largest category (887 messages), whereas caution and advice messages are the least represented (35 messages). Such imbalance may bias classifiers towards frequent categories. To address this issue, balanced class weights are used for Logistic Regression, while BERT is trained with weighted cross-entropy loss.

Fig. 1. Overall NLP–routing architecture for the Nepal earthquake case study.

Messages are classified into nine humanitarian categories and mapped to four routing priorities, from P1 (highest urgency) to P4 (lowest urgency). Ground-truth and predicted priorities are retained separately in the simulation message schedule. Four routing conditions are evaluated: standard Binary Spray and Wait without semantic priorities, oracle priority routing using ground-truth priorities, Logistic Regression priority routing, and BERT priority routing. The same message schedule, Kathmandu road map, node configuration, and mobility seeds are used across conditions to support a controlled comparison of network performance and the effects of classification errors.

C. Nepal Earthquake Message Preprocessing and Classification Design The study uses an English-language social media dataset collected during the 2015 Nepal earthquake. After preprocessing, 3,003 messages are retained and divided into training and test sets using a fixed stratified 80:20 split with random seed 42, yielding 2,402 training messages and 601 test messages. The same partition is used for both classifiers to ensure a consistent comparison, as summarised in Table I. The dataset is publicly available at (https://crisisnlp.qcri.org/lrec2016/lrec2016.html)

Fig. 2. Distribution of humanitarian message categories in the cleaned Nepal earthquake dataset.

Text preprocessing is performed to standardise the Nepal earthquake social media messages before classification. The workflow includes URL and user-mention replacement, hashtag normalisation, whitespace normalisation, and duplicate removal. Both original and cleaned messages are retained alongside message identifiers and ground-truth labels to support traceability and subsequent classification error analysis. The complete preprocessing workflow is illustrated in Fig. 3.

Weighted cross-entropy loss is used to address class imbalance without over-sampling or under-sampling. Both classifiers produce predicted humanitarian categories and associated confidence scores, which are subsequently converted into routing priorities. D. Semantic Priority Mapping and NLP–Simulator Integration Design

Fig. 3. Text preprocessing workflow for the Nepal earthquake social media messages.

TABLE II NLP C LASSIFIER C ONFIGURATION Setting

Logistic Regression

BERT

Input representation N-gram range Minimum document frequency Maximum document frequency Sublinear TF scaling Maximum iterations Pretrained model Maximum sequence length Batch size Training epochs Learning rate Weight decay Warm-up proportion Class imbalance handling

TF–IDF vectors (1, 2) 2

Token sequences – –

0.9

–

Enabled 2,000 – –

– – bert-base-uncased 128

– – – – – Balanced class weights

8 3 2 × 10−5 0.01 10% Weighted cross-entropy

Humanitarian message categories describe the semantic content of a message, whereas routing priorities represent its relative urgency for delivery. A fixed mapping is therefore defined to convert the nine humanitarian categories into four routing priority levels, from P1 (highest urgency) to P4 (lowest urgency). The same mapping is applied to ground-truth and predicted categories to ensure a consistent comparison. The mapping assigns messages concerning injured or dead people and missing, trapped, or found people to P1. Displacement and evacuation information, infrastructure damage, and caution and advice are assigned to P2. Donation and volunteering messages and other useful information are assigned to P3, while sympathy and emotional support and irrelevant messages are assigned to P4. The complete mapping is summarised in Table III. TABLE III M APPING FROM H UMANITARIAN C ATEGORIES TO ROUTING P RIORITIES Humanitarian Category

The main configurations of the two classifiers are summarised in Table II. Two classifiers are employed for nine-class humanitarian message classification. The first combines TF– IDF feature extraction with Logistic Regression to provide a conventional machine-learning baseline. The vectoriser uses unigram and bigram features, excludes terms occurring in fewer than two documents or more than 90% of documents, and applies sublinear term-frequency scaling. Logistic Regression uses balanced class weights to account for category imbalance, with a maximum of 2,000 optimisation iterations. The second classifier fine-tunes the pretrained BERT-baseuncased model using the same training and test partition. Messages are tokenised and padded or truncated to a maximum length of 128 tokens. Fine-tuning is conducted for three epochs with a batch size of 8, a learning rate of 2 × 10−5 , a weight decay of 0.01, and a 10% learning-rate warm-up proportion.

Injured or dead people Missing, trapped, or found people Displaced people and evacuations Infrastructure and utilities damage Caution and advice Donation needs, offers, or volunteering services Other useful information Sympathy and emotional support Not related or irrelevant

Level

Priority

High High Medium-high Medium-high Medium-high Medium

P1 P1 P2 P2 P2 P3

Medium Low Low

P3 P4 P4

The notation used to formalise the semantic priority mapping and its integration into the routing framework is summarised in Table IV. TABLE IV M AIN N OTATION U SED IN THE PAPER Notation

Meaning

i C yi (q) ŷi q π(c) ptrue i pred,(q) pi proute i

Index of a disaster message. Set of nine humanitarian message categories. Ground-truth humanitarian category of message i. Category predicted for message i by classifier q. NLP classifier, where q ∈ {LR, BERT}. Fixed mapping from category c to routing priority. Priority derived from the ground-truth category. Priority derived from the category predicted by classifier q. Active routing priority used during simulation.

Let C denote the set of nine humanitarian message categories. Each message i has a ground-truth category yi ∈ C, which is mapped to a routing priority through the predefined

mapping function π. The ground-truth routing priority is defined as ptrue = π(yi ). (1) i Here, π is the fixed mapping from humanitarian categories to the four routing priorities, formally defined as π : C → {P1, P2, P3, P4}. For classifier q, the predicted routing priority is defined as   pred,(q) (q) pi = π ŷi , q ∈ {LR, BERT}. (2) (q)

Here, ŷi ∈ C denotes the category predicted by classifier q, where q represents either Logistic Regression (LR) or BERT. Both ground-truth and predicted categories are mapped using the same function π. Lower priority indices indicate greater urgency. Keeping ground-truth and predicted priorities separate allows classification errors to be distinguished from their downstream routing effects. The classification outputs are then integrated into the network simulation through a CSV-based message schedule. For each message, the schedule retains the ground-truth priority and the priorities predicted by the NLP classifiers. The custom PriorityMessageEventGenerator reads the message schedule and transfers the corresponding priority attributes, truePriority and predictedPriority, to the generated messages in The ONE simulator. This design separates message classification from routing decisions while allowing ground-truth and predicted priorities to be evaluated under the same simulation conditions. Four routing conditions are considered in the simulation. Standard Binary Spray and Wait serves as the baseline without semantic priority information. Oracle priority routing uses the ground-truth priority, whereas the Logistic Regression and BERT conditions use their respective predicted priorities. For the priority-aware conditions, the active routing priority is therefore selected as  true  Oracle, pi , route pi = ppred,(LR) (3) , LR-based routing, i   pred,(BERT) pi , BERT-based routing. The standard Binary Spray and Wait baseline does not use semantic priorities in its forwarding decisions. This comparison distinguishes the potential benefits of priority-aware routing from the effects of classification errors. The active priority is made available to the priority-aware routing logic to support transmission scheduling, buffer admission and dropping, and message copy management. Messages with different urgency levels can therefore receive different treatment during forwarding and resource allocation. Under the oracle condition, routing decisions are based on groundtruth priorities, providing a reference for assessing the effects of using NLP-predicted priorities. The Logistic Regression and BERT conditions are compared against this reference to examine how classification errors affect the delivery of urgent messages under constrained network conditions.

E. Kathmandu Earthquake Network Scenario Design The network scenario is designed as a controlled case study of post-earthquake emergency communication in Kathmandu. It represents a situation in which conventional communication infrastructure is partially unavailable and mobile devices rely on intermittent peer-to-peer contacts to exchange messages. The purpose of the scenario is not to reproduce the communication conditions of the 2015 Nepal earthquake exactly. Instead, the Kathmandu road network, disaster-related message data, and representative emergency-response roles are combined to create a re producible urban opportunistic-network scenario. The Kathmandu emergency communication scenario is implemented in The ONE Simulator using the configured road map, node groups, mobility settings and wireless interfaces. Fig. 4 shows the implemented scenario during execution. The visualisation is used to confirm that the map, mobile nodes, and opportunistic contacts are loaded correctly.

Fig. 4. Execution of the Kathmandu earthquake communication scenario in the ONE simulator.

The simulation is conducted in The ONE simulator using a Kathmandu road map with a world size of 3268×2482 metres. Node movement is restricted to the road network using the ShortestPathMapBasedMovement model. The scenario contains 95 mobile nodes divided into three groups: residents, rescue workers, and rescue vehicles. Residents act as message sources, while rescue workers serve as message destinations and mobile relays. Rescue vehicles move at higher speeds and provide additional forwarding opportunities between otherwise separated parts of the network. The three node groups have different movement speeds and buffer capacities to represent their different operational roles and communication resources. Residents have the smallest buffers, whereas rescue vehicles have larger buffers and higher mobility. These differences introduce heterogeneous resource constraints into the network. Nevertheless, connectivity remains intermittent because messages can be exchanged only when nodes move within wireless transmission range. The simulation runs for 43,200 seconds (12 hours), with a transmission range of 50 metres and a transmission speed of 50 KB/s. Message sizes range from 100 KB to 500 KB, and message creation intervals are randomly selected between 30 and 60 seconds. The high-load configuration generates 959

offered messages per simulation run, with residents serving as sources and rescue workers as destinations. The node-group configurations, including mobility speeds and buffer capacities, are summarised in Table V, while the main communication and traffic parameters are listed in Table VI. TABLE V N ODE G ROUPS U SED IN THE K ATHMANDU E ARTHQUAKE S CENARIO Group Residents Rescue workers Rescue vehicles

Nodes Addresses Speed (m/s) Buffer 60 20 15

0–59 60–79 80–94

0.3–1.2 1.0–2.5 5.0–12.0

5 MB 20 MB 50 MB

TABLE VI K ATHMANDU E ARTHQUAKE S CENARIO M AIN C OMMUNICATION AND T RAFFIC PARAMETERS Parameter

Value

Simulation duration Transmission range Transmission speed Message size Message interval Offered messages Message sources Message destinations Mobility model

43,200 s 50 m 50 KB/s 100–500 KB 30–60 s 959 Resident nodes 0–59 Rescue-worker nodes 60–79 ShortestPathMapBasedMovement

This configuration is intended to create competition for temporary contact opportunities and limited buffer space. It therefore provides a controlled setting in which to examine how semantic priority affects message transmission scheduling, buffer management, and the delivery of urgent humanitarian information. The same network configuration and message schedule are retained across routing conditions so that differences in performance can be attributed primarily to the routing strategy and its use of priority information. F. Priority-Aware Routing and Buffer Management Design The proposed method incorporates semantic priorities into two routing operations: transmission selection and buffer management. The following algorithms describe how message priorities influence forwarding order and buffer eviction, while retaining the original Binary Spray and Wait copy management mechanism. Priority is introduced into the relay transmission process by modifying the order in which candidate messages are considered. During each routing update, direct delivery to the final destination is attempted before any relay transmission. This preserves the normal behaviour of Spray and Wait when the destination is directly reachable. If direct delivery is not possible, the router identifies buffered messages that still contain more than one copy and are therefore eligible for the spray phase. The existing queue mode is first applied to these messages. The resulting list is

then reordered according to the active routing priority, with P1 considered before P2, followed by P3 and P4. The priority sort is stable. As a result, messages with the same priority retain the ordering produced by the original queue mechanism. Priority therefore acts as the primary scheduling criterion without removing the existing tie breaking behaviour of the ONE. Algorithm 1 summarises this process. Algorithm 1 Priority Based Transmission Selection Require: Buffered messages M , available connections C Ensure: Transmission candidate ordering 1: if a message can be delivered directly to its destination then 2: Transmit the directly deliverable message 3: return 4: end if 5: Ms ← {m ∈ M | remainingCopies(m) > 1} 6: Apply the standard queue ordering to Ms 7: Stable-sort Ms by ascending routing priority 8: for all messages m ∈ Ms do 9: Attempt transmission over an available connection in C 10: end for

When buffer space is insufficient, the router considers eligible lower-priority messages for eviction before admitting an incoming message. Messages currently being transmitted are excluded from eviction. If sufficient space cannot be released without violating the priority constraints, the incoming message is rejected. Algorithm 2 summarises the buffer management procedure. Algorithm 2 Priority-Aware Buffer Admission and Eviction Require: Incoming message min , buffer B, required free space S Ensure: Admission or rejection of min 1: pin ← p(min ), E ← ∅ 2: for all m ∈ B do 3: if m is not being transmitted and p(m) ≥ pin then 4: E ← E ∪ {m} 5: end if 6: end for 7: Sort E by descending priority; break ties by earliest receive time 8: Esel ← ∅, Sfree ← 0 9: for all m in sorted E do 10: if Sfree < S then 11: Esel ← Esel ∪ {m} 12: Sfree ← Sfree + size(m) 13: end if 14: end for 15: if Sfree < S then 16: Reject min 17: return 18: end if 19: for all m ∈ Esel do 20: Remove m from the buffer 21: end for 22: Admit min

IV. E VALUATION A. Evaluation Setup The evaluation consists of two complementary experiments: NLP classification and network routing. For NLP evaluation, Logistic Regression and BERT are tested on the same fixed test set of 601 Nepal earthquake messages without retraining. Classification performance is assessed at both the original nine-category level and the mapped four-level routing priority level. For network evaluation, four routing conditions are compared: Standard Binary Spray and Wait, Oracle, Logistic Regression, and BERT. All conditions use the same underlying message traffic, Kathmandu network scenario, and mobility configuration. Message creation times, source and destination nodes, message sizes, and ground-truth priorities remain unchanged. Only the source of the active routing priority differs between the priority-aware conditions. The four conditions and their corresponding priority sources are summarised in Table VII.

Network performance is evaluated across the four routing conditions, with particular attention to messages whose ground-truth priority is P1. P1 delivery probability measures the proportion of created P1 messages successfully delivered to their destinations. Average and median P1 latency are calculated over successfully delivered P1 messages. Message drops measure the total number of messages removed due to buffer capacity limits, while P1 buffer loss counts groundtruth P1 messages lost under buffer pressure. These metrics assess both delivery performance and the protection of urgent messages. The evaluation metrics used at the humanitarian classification, routing priority and network levels are summarised in Table VIII. TABLE VIII S UMMARY OF E VALUATION M ETRICS U SED IN THE S TUDY Evaluation Level Metric Humanitarian Classification Accuracy Macro-F1

TABLE VII ROUTING C ONDITIONS U SED IN THE E VALUATION Weighted-F1

Condition Standard Oracle Logistic Regression BERT

Priority Used None Ground truth LR prediction BERT prediction

Purpose Baseline Ideal reference Predicted priority Predicted priority

Routing Priority

Priority Accuracy Priority Macro-F1 P1 Recall

Each routing condition is evaluated using five mobility seeds, resulting in 20 simulation runs in total. The same seeds are reused across all conditions to ensure identical mobility trajectories and comparable contact opportunities. For network evaluation, messages are grouped by their groundtruth priorities rather than the priorities used during routing. Therefore, a genuine P1 message remains in the P1 evaluation group even if an NLP classifier assigns it a lower priority. This allows the network effects of classification errors to be evaluated directly.

Under-prioritisation Rate Severe Priority Error Rate Network

P1 Delivery Probability Average P1 Latency Median P1 Latency Message Drops

B. Evaluation Metrics The performance of the proposed system is measured from humanitarian classification, priority prediction, and network performance. Accuracy, macro-F1, and weighted-F1 are provided for nine humanitarian classifications. Macro-F1 considers all the classifications equally while weighted-F1 considers the sizes of samples in each classification. For priority prediction, priority accuracy, priority macro-F1 and P1 recall are used to measure the performance in four priority levels. Two measures are also provided to indicate the impact of priority prediction errors in operations, namely under-priority rate and severe priority error rate. Under-priority means the priority of a message is predicted to be lower than its ground-truth priority. Severe priority error means the predicted priority and ground-truth priority are different by at least two levels. P1 recall is significant since P1 is the most urgent priority level

P1 Buffer Loss

Purpose Overall proportion of correctly classified humanitarian messages. Measures class-balanced classification performance by giving equal importance to all nine categories. Measures overall F1 while accounting for the number of samples in each category. Proportion of messages assigned to the correct P1–P4 priority level. Measures balanced performance across the four routing priority levels. Proportion of ground-truth P1 messages correctly predicted as P1. Proportion of messages assigned a less urgent priority than their ground-truth priority. Proportion of predictions differing from the ground-truth priority by two or more levels. Proportion of created ground-truth P1 messages successfully delivered to their destinations. Mean delivery delay of successfully delivered ground-truth P1 messages. Median delivery delay of successfully delivered ground-truth P1 messages. Number of messages removed because of limited buffer capacity. Number of ground-truth P1 messages lost under buffer pressure.

C. Nepal Earthquake NLP Performance The performance of the two NLP models is tested separately without considering the network simulation. The two models are tested on the same test set of 601 messages from the Nepal earthquake. Their performance in terms of humanitarian classification and priority prediction is summarized in Table IX. BERT performs better than Logistic Regression in all nine humanitarian classifications. The accuracy goes up from 0.6772 to 0.7188 and weighted F1 goes up from 0.6748 to 0.7096. Macro F1 goes up from 0.5549 to 0.5838, which means the balanced classification performance is better. For both NLP models, macro F1 is lower than weighted F1, which means the classification performance is not balanced

among the nine classifications and there might still be some classifications that are less common or more difficult to classify correctly. Although BERT performs better in overall, there are still some gaps, meaning that the humanitarian classification is not perfect yet. TABLE IX OVERALL NLP C LASSIFICATION AND ROUTING P RIORITY P ERFORMANCE ON THE F IXED T EST S ET

Logistic Metric Regression BERT Humanitarian category accuracy 0.6772 0.7188 Humanitarian category macro F1 0.5549 0.5838 Humanitarian category weighted F1 0.6748 0.7096 Priority accuracy 0.7804 0.8070 Priority macro F1 0.7256 0.7560 P1 recall 0.8302 0.8868 Under-prioritisation rate 0.1048 0.0899 Severe priority error rate 0.0632 0.0349

After mapping the predicted humanitarian categories to four routing priorities, both classifiers achieve higher accuracy than in the original nine-category task, as multiple categories may share the same routing priority. BERT outperforms Logistic Regression in priority accuracy (0.8070 vs. 0.7804), priority macro F1 (0.7560 vs. 0.7256), and P1 recall (0.8868 vs. 0.8302). It also reduces the underprioritisation rate from 0.1048 to 0.0899 and the severe priority error rate from 0.0632 to 0.0349, indicating fewer errors that may compromise urgent message handling. Overall, BERT provides more reliable priority predictions. The following network evaluation examines whether these improvements translate into better delivery performance for urgent messages. D. Nepal Earthquake Priority Error Analysis Overall priority accuracy does not fully reveal how classification errors are distributed across the four routing priorities. Fig. 5 compares the priority confusion matrices of Logistic Regression and BERT to identify differences in their classification errors, particularly for urgent P1 messages.

Fig. 5. Priority confusion matrices for Logistic Regression and BERT.

For P1 ground-truth messages, Logistic Regression correctly classifies 88 out of 106 messages while BERT correctly

classifies 94 messages. BERT also decreases the number of P1 messages being incorrectly classified to P3/P4 from 15 to 7. However, the number being incorrectly classified to P2 increases from 3 to 5, meaning that the improvement is not consistent for all types of errors. From the confusion matrices, it can be seen that BERT improves the classification of urgent messages, but there are still some errors in priority classification. These results lead to the evaluation of the network in the following section to see if the improved priority prediction can lead to better performance in delivering urgent messages.

E. Kathmandu Priority Routing Performance The network evaluation examines whether semantic priority improves the treatment of urgent messages under the fixed Kathmandu high load scenario. Results are aggregated across the five mobility seeds for each routing condition. The analysis focuses primarily on messages whose ground truth priority is P1. This ensures that urgent traffic is evaluated according to its actual humanitarian importance, regardless of the priority assigned by an NLP classifier during routing. Four conditions are compared: standard Binary Spray and Wait, Oracle priority routing, Logistic Regression predicted priority routing and BERT predicted priority routing. Table X summarises the P1 delivery and latency results. The delivery probability and latency values summarise performance across the five mobility seeds, while Total P1 Delivered reports the cumulative number of successfully delivered ground truth P1 messages across the five runs. TABLE X P1 DELIVERY AND LATENCY PERFORMANCE ACROSS THE FIVE MOBILITY SEEDS . P1 Delivery P1 Delivered Average Latency Median Latency Condition Probability Total (s) (s) Standard 0.9586 810 1835.68 1392.6 Oracle 0.9586 810 1826.23 1378.0 Logistic Regression 0.9527 805 1839.80 1377.9 BERT 0.9574 809 1842.80 1379.6

P1 delivery probability remains high across all four routing conditions. Standard and Oracle both achieve a delivery probability of 0.9586, delivering 810 ground-truth P1 messages across the five mobility seeds. Logistic Regression achieves a lower probability of 0.9527, with 805 messages delivered, while BERT achieves 0.9574, delivering 809 messages. Compared with Logistic Regression, BERT improves P1 delivery performance. However, it does not outperform Standard or Oracle. This suggests that priority prediction alone does not guarantee higher delivery probability under the evaluated scenario. Fig. 6 compares the P1 delivery probabilities across the four routing conditions. The error bars show the variation across the five mobility seeds.

buffer loss compared with Logistic Regression. These results suggest that BERT improves urgent message protection over Logistic Regression, although it does not fully match the Oracle condition. TABLE XI B UFFER LOSSES BY GROUND - TRUTH PRIORITY ACROSS PRIORITY- AWARE ROUTING CONDITIONS .

Condition P1 P2 P3 P4 Total Oracle 0 0 0 73 73 Logistic Regression 5 0 16 53 74 BERT 1 1 11 52 65 Fig. 6. P1 delivery probability across the four routing conditions.

Delivery latency is analysed to compare how quickly successfully delivered P1 messages reach their destinations. Oracle achieves the lowest average latency of 1826.23 s, while BERT records the highest at 1842.80 s. Fig.7 shows the variation in P1 delivery latency across the five mobility seeds under the four routing conditions.

Fig. 8. Buffer losses by ground-truth priority across priority-aware routing conditions. Fig. 7. P1 delivery latency across four routing conditions and five mobility seeds.

For median latency, Logistic Regression achieves the lowest value of 1377.9 s, closely followed by Oracle at 1378.0 s. Standard has the highest median latency of 1392.6 s, while BERT records 1379.6 s. Overall, no single routing condition achieves the best result across both latency metrics. Although BERT improves P1 delivery probability compared with Logistic Regression, it does not consistently reduce delivery latency. These results highlight the need to evaluate delivery success and latency together. Delivery probability and latency do not fully describe the effect of priority-aware routing under congestion. It is also necessary to examine which messages are lost when buffer resources become limited. Therefore, Table XI and Fig. 8 compare buffer losses by ground-truth priority across the three priority-aware routing conditions. The Standard condition is excluded because the reported buffer loss statistics are derived from the priority-aware buffer management mechanism. The results show that BERT produces the lowest total buffer loss, with 65 messages lost, compared with 73 for Oracle and 74 for Logistic Regression. However, Oracle provides the strongest protection for urgent messages, with no P1 messages lost. Logistic Regression loses five P1 messages, whereas BERT loses only one. This represents an 80% reduction in P1

The preceding sections evaluated NLP classification and network routing separately. This section combines their results to examine whether improved priority prediction leads to better protection of urgent messages. Table XII compares Logistic Regression and BERT across classification and network performance metrics. TABLE XII E ND - TO - END COMPARISON OF L OGISTIC R EGRESSION AND BERT. Metric Priority Accuracy P1 Recall Under-prioritisation Rate Severe Priority Error Rate P1 Delivery Probability Total P1 Delivered Average P1 Latency (s) Median P1 Latency (s) P1 Buffer Loss

Logistic Regression 0.7804 0.8302 0.1048 0.0632 0.9527 805 1839.80 1377.9 5

BERT 0.8070 0.8868 0.0899 0.0349 0.9574 809 1842.80 1379.6 1

At the classification level, BERT achieves higher priority accuracy (0.8070 vs. 0.7804) and P1 recall (0.8868 vs. 0.8302), while reducing under-prioritisation and severe priority errors. Fig. 9 summarises the overall comparison using normalised performance metrics.

B. Research Limitations

Fig. 9. Parallel coordinates comparison of Logistic Regression and BERT.

At the network level, the benefits are mixed. BERT slightly improves P1 delivery probability from 0.9527 to 0.9574 and delivers four additional P1 messages. However, average and median latency remain similar, with no clear reduction in delivery delay. The most notable improvement is buffer protection: P1 message losses decrease from five under Logistic Regression to one under BERT. Overall, more accurate priority classification improves the protection of urgent messages under buffer pressure, although mobility and contact availability continue to limit network performance. V. C ONCLUSION AND F UTURE W ORK A. Conclusion This paper presents the analysis of NLP-based humanitarian messages’ classification for priority-aware routing in opportunistic networks. The classification of Nepal earthquake messages with Logistic Regression and BERT models was analysed in terms of their priority prediction in network simulations. It can be seen that BERT provides higher classification results in terms of priority accuracy and P1 recall, as well as lower rates of under-priority and severe priority errors, compared to Logistic Regression. Network level, these results result in higher protection of urgent messages in case of buffer pressure. Specifically, the rate of P1 buffer loss is reduced from five to one for BERT compared to Logistic Regression. Nevertheless, the classification results do not result in corresponding improvements in network level results as only small improvement in P1 delivery probability is observed while the delivery latency is the same for both models. Therefore, it can be concluded that correct semantic priority prediction can help in improving the handling of urgent humanitarian messages, however, the delivery performance cannot be improved significantly due to the nature of the network mobility, contact and resource limitations. In general, the paper shows the potential of combining NLP-based priority prediction with priority-aware routing and buffer handling. Moreover, it is important to note that the classification models should be analysed in terms of network level effects, rather than classification results.

The limitations of the work include the following: Firstly, the NLP models are tested only on social media content of the 2015 Nepal earthquake, and hence, the performance cannot be generalized to other disasters, platforms, and languages. Secondly, the data is labeled with humanitarian categories but not independently with urgency levels. Hence, the P1P4 prioritization is based on the mapping of categories to priorities in this work, and different mappings can lead to different routing. Thirdly, the network simulation is performed in a controlled environment of Kathmandu with few mobility seeds and Binary Spray and Wait routing protocol; thus, the performance can vary in other environments. Lastly, the routing approach does not take into account the confidence of the prediction while assigning discrete P1-P4 priorities, and hence, different uncertainties can be treated the same way. Thus, the results of the work can be considered as performance in a controlled environment of Nepal earthquake only. C. Future Work The proposed NLP models can be tested on other disaster data, social media, and languages. Other category-to-priority assignments should be tested with humanitarian and emergency response professionals. Confidence in predictions can be used in transmission scheduling and buffer control for uncertainty-aware routing. Other network aspects such as routing protocols, traffic conditions, buffer sizes, transmission distances, and mobility patterns can also be considered. Moreover, We aim to incorporate protocol into the MODiToNeS platform and deploy it in the next generation of rescue services communications [46]. Lastly, realistic mobility/contact traces can be used to assess if the advantages found in the Kathmandu environment are generalisable. R EFERENCES [1] Y. Jahir, M. Atiquzzaman, H. Refai, A. Paranjothi, and P. G. LoPresti, “Routing protocols and architecture for disaster area network: A survey,” Ad Hoc Networks, vol. 82, p. 1–14, 1 2019. [Online]. Available: https://www.sciencedirect.com/science/article/pii/S1570870518305626? casa\ token=KjxNQezCbVgAAAAA:uFnauaSSf1k4BbWjd4tHTOiffk 9ni2I-LeHnEidF5cu-6RyYqV4TjCwMHcY6CyCEOqaAVen4HIK6 [2] M. Matracia, N. Saeed, M. A. Kishk, and M.-S. Alouini, “Postdisaster communications: Enabling technologies, architectures, and open challenges,” IEEE Open Journal of the Communications Society, vol. 3, pp. 1177–1205, 2022. [3] S. CC, V. Raychoudhury, G. Marfia, and A. Singla, “A survey of routing and data dissemination in delay tolerant networks,” Journal of Network and Computer Applications, vol. 67, pp. 128–146, 5 2016. [4] K. Fall, “A delay-tolerant network architecture for challenged internets,” Proceedings of the 2003 conference on Applications, technologies, architectures, and protocols for computer communications - SIGCOMM ’03, 2003. [5] M. Imran, P. Mitra, and C. Castillo, “Twitter as a lifeline: Humanannotated twitter corpora for nlp of crisis-related messages,” ACLWeb, 5 2016. [Online]. Available: https://aclanthology.org/L16-1259/ [6] N. Rani and A. Malik, “Resource-aware routing in opportunistic networks: existing protocols and open research issues,” International Journal of Cloud Computing, vol. 12, no. 2/3/4, p. 367, 2023. [7] A. Roy, T. Acharya, and S. DasBit, “Quality of service in delay tolerant networks: A survey,” Computer Networks, vol. 130, pp. 121–133, 1 2018.

[8] F. Alam, U. Qazi, M. Imran, and Ferda Ofli, “Humaid: Human-annotated disaster incidents data from twitter with deep learning benchmarks,” Proceedings of the International AAAI Conference on Web and Social Media, vol. 15, pp. 933–942, 5 2021. [9] F. Alam, H. Sajjad, M. Imran, and F. Ofli, “Crisisbench: Benchmarking crisis-related social media datasets for humanitarian information processing,” Proceedings of the International AAAI Conference on Web and Social Media, vol. 15, pp. 923–932, 5 2021. [10] M. Basit, B. Alam, Z. Fatima, and S. Shaikh, “Natural disaster tweets classification using multimodal data,” in Proceedings of the 2023 conference on empirical methods in natural language processing, 2023, pp. 7584–7594. [11] E. Wang, Y.-J. Yang, J. Wu, and W.-B. Liu, “A buffer scheduling method based on message priority in delay tolerant networks,” Journal of Computer Science and Technology, vol. 31, no. 6, pp. 1228–1245, 11 2016. [12] K. Shin, K. Kim, and S. Kim, “Traffic management strategy for delaytolerant networks,” Journal of Network and Computer Applications, vol. 35, no. 6, pp. 1762–1770, 11 2012. [13] “Crisisnlp,” Qcri.org, 2016. [Online]. Available: https://crisisnlp.qcri.o rg/lrec2016/lrec2016.html [14] J. Devlin, M.-W. Chang, K. Lee, and K. Toutanova, “Bert: Pre-training of deep bidirectional transformers for language understanding,” in Proceedings of the 2019 conference of the North American chapter of the association for computational linguistics: human language technologies, volume 1 (long and short papers), 2019, pp. 4171–4186. [15] T. Spyropoulos, K. Psounis, and C. S. Raghavendra, “Spray and wait,” Proceeding of the 2005 ACM SIGCOMM workshop on Delay-tolerant networking - WDTN ’05, 2005. [16] A. Keränen, J. Ott, and T. Kärkkäinen, “The one simulator for dtn protocol evaluation,” in Proceedings of the 2nd international conference on simulation tools and techniques, 2009, pp. 1–10. [17] H. Chenji and R. Stoleru, “Delay-tolerant networks (dtns) for emergency communications.” Woodhead Publishing, 1 2021, pp. 103–134. [Online]. Available: https://www.sciencedirect.com/science/ch apter/edited-volume/abs/pii/B9780081027936000060 [18] A. Martı́n-Campillo, J. Crowcroft, E. Yoneki, and R. Martı́, “Evaluating opportunistic networks in disaster scenarios,” Journal of Network and Computer Applications, vol. 36, no. 2, pp. 870–880, 3 2013. [19] R. Sachdeva and A. Dev, “Review of opportunistic network: Assessing past, present, and future,” International Journal of Communication Systems, 5 2021. [20] L. Pelusi, A. Passarella, and M. Conti, “Opportunistic networking: data forwarding in disconnected mobile ad hoc networks,” IEEE Communications Magazine, vol. 44, no. 11, pp. 134–141, 11 2006. [Online]. Available: https://ieeexplore.ieee.org/abstract/document/4014 485 [21] A. Vahdat, D. Becker et al., “Epidemic routing for partially connected ad hoc networks,” 2000. [22] A. Lindgren, A. Doria, and O. Schelén, “Probabilistic routing in intermittently connected networks,” ACM SIGMOBILE Mobile Computing and Communications Review, vol. 7, no. 3, pp. 19–20, 7 2003. [23] E. Rosas, F. Garay, and N. Hidalgo, “Context-aware self-adaptive routing for delay tolerant network in disaster scenarios,” Ad Hoc Networks, vol. 102, p. 102095, 5 2020. [24] M. Radenkovic and A. Grundy, “Congestion aware forwarding in delay tolerant and social opportunistic networks,” 2011 Eighth International Conference on Wireless On-Demand Network Systems and Services, pp. 60–67, 1 2011. [25] M. Radenkovic and A. Grundy, “Efficient and adaptive congestion control for heterogeneous delay-tolerant networks,” Ad Hoc Networks, vol. 10, no. 7, pp. 1322–1345, 4 2012. [Online]. Available: https://www.sciencedirect.com/science/article/pii/S1570870512000637 [26] T. Sakano, Z. M. Fadlullah, T. Ngo, H. Nishiyama, M. Nakazawa, F. Adachi, N. Kato, A. Takahara, T. Kumagai, H. Kasahara, and S. Kurihara, “Disaster-resilient networking: a new vision based on movable and deployable resource units,” IEEE Network, vol. 27, no. 4, pp. 40–46, 2013. [27] S. Basu, S. Roy, and S. Das Bit, Reliable Post Disaster Services over Smartphone Based DTN. Springer, 3 2019. [28] V. S. Ha Huynh and M. Radenkovic, “A novel cross-layer framework for large scale emergency communications,” 2017 13th International Wireless Communications and Mobile Computing Conference (IWCMC), pp. 2152–2157, 6 2017.

[29] M. Radenkovic and V. S. H. Huynh, “Collaborative cognitive content dissemination and query in heterogeneous mobile opportunistic networks,” Proceedings of the 3rd Workshop on Experiences with the Design and Implementation of Smart Objects, pp. 7–12, 10 2017. [30] M. Radenkovic and V. S. H. Huynh, “Cognitive caching at the edges for mobile social community networks: A multi-agent deep reinforcement learning approach,” IEEE Access, vol. 8, pp. 179 561–179 574, 2020. [31] V. S. H. Huynh, M. Radenkovic, and N. Wang, “Distributed spatialtemporal demand and topology aware resource provisioning for edge cloud services,” 2021 Sixth International Conference on Fog and Mobile Edge Computing (FMEC), pp. 1–8, 12 2021. [32] M. Radenkovic and V. S. Ha Huynh, “Energy-aware opportunistic charging and energy distribution for sustainable vehicular edge and fog networks,” 2020 Fifth International Conference on Fog and Mobile Edge Computing (FMEC), pp. 5–12, 4 2020. [33] M. Radenkovic and A. Walker, “Cognitivecharge,” Proceedings of the 4th ACM MobiHoc Workshop on Experiences with the Design and Implementation of Smart Objects, pp. 1–9, 6 2018. [34] A. Krifa, C. Barakat, and T. Spyropoulos, “Message drop and scheduling in dtns: Theory and practice,” IEEE Transactions on Mobile Computing, vol. 11, no. 9, pp. 1470–1483, 9 2012. [35] S. Singh and A. Verma, “Buffer management techniques in delay tolerant networks: A comprehensive survey,” Recent Advances in Computer Science and Communications, vol. 16, no. 8, 10 2023. [36] M. Imran, C. Castillo, F. Diaz, and S. Vieweg, “Processing social media messages in mass emergency,” ACM Computing Surveys, vol. 47, no. 4, pp. 1–38, 6 2015. [Online]. Available: https://dl.acm.org/citation.cfm?id=2771588 [37] M. Imran, C. Castillo, J. Lucas, P. Meier, and S. Vieweg, “Aidr,” Proceedings of the 23rd International Conference on World Wide Web - WWW ’14 Companion, 2014. [38] K. R. Chowdhary, “Natural language processing,” Fundamentals of Artificial Intelligence, pp. 603–649, 2020. [39] Mikael Brunila, R. Zhao, Andrei Mircea, S. Lumley, and R. Sieber, “Bridging the gap between supervised classification and unsupervised topic modelling for social-media assisted crisis management,” ACL Anthology, pp. 33–49, 4 2021. [Online]. Available: https://aclanthology .org/2021.adaptnlp-1.5/ [40] J. Liu, T. Singhal, L. T. Blessing, K. L. Wood, and K. H. Lim, “Crisisbert: a robust transformer for crisis classification and contextual crisis embedding,” in Proceedings of the 32nd ACM conference on hypertext and social media, 2021, pp. 133–141. [41] H. Li, D. Caragea, C. Caragea, and N. Herndon, “Disaster response aided by tweet classification with a domain adaptation approach,” Journal of Contingencies and Crisis Management, vol. 26, no. 1, pp. 16–27, 2018. [42] P. Seeberger and K. Riedhammer, “Enhancing crisis-related tweet classification with entity-masked language modeling and multitask learning,” Proceedings of the Second Workshop on NLP for Positive Impact (NLP4PI), pp. 70–78, 2022. [Online]. Available: https://aclanthology.org/2022.nlp4pi-1.9/ [43] M. Sokolova and G. Lapalme, “A systematic analysis of performance measures for classification tasks,” Information processing \& management, vol. 45, no. 4, pp. 427–437, 2009. [44] S. Bhattacharjee, S. Basu, S. Roy, and S. D. Bit, “Best-effort delivery of emergency messages in post-disaster scenario with content-based filtering and priority-enhanced prophet over dtn,” in 2016 8th International Conference on Communication Systems and Networks (COMSNETS). IEEE, 2016, pp. 1–7. [45] E. Rosas, O. Andrade, and N. Hidalgo, “Effective communication for message prioritization in dtn for disaster scenarios,” Peer-to-Peer Networking and Applications, vol. 16, no. 1, pp. 368–382, 2023. [46] M. Radenkovic, J. Crowcroft, and M. H. Rehmani, “Towards low cost prototyping of mobile opportunistic disconnection tolerant networks and systems,” IEEE Access, vol. 4, pp. 5309–5321, 2016.

Related documents

• NLP-based Message Prioritisation for Opportunistic Routing: A Case Study of Nepal Earthquake Emergency Communication#1488725
• Maui wildfire evacuation decision prediction using domain-knowledge-guided large language models#1346037
• Large Language Models in Transportation Systems Management and Operations: From Text Reasoning to Multi-modal Decision Support#1386293
• When Nouns Turn Negative: The Evaluative Absorption Model#204039
• Semantic burstiness across principal directions of textual variation#1457011