Enhancing Few-Shot Text Classification with Parameter-Efficient Tuning of Large Language Models

Abstract

Traditional few-shot text classification models focus only on label prediction and cannot extract structured information such as entities or events, limiting their usefulness in real-world, semantics-driven tasks. They also rarely use external knowledge or parameter-efficient tuning, leading to shallow representations and weaker performance. To address this, this paper proposes a knowledge-aware multi-task framework that integrates few-shot classification with entity and event extraction. A single BERT encoder with IA3 adapters enables efficient tuning, while semantic triples extracted via spaCy and aligned with WordNet and ConceptNet are encoded using TransE. A BiLSTM captures sequential context and a softmax decoder performs token-level prediction. Experiments show strong results—97.97% accuracy, 98.00% precision, 97.95% recall, and 97.96% F1—surpassing state-of-the-art baselines. Ablation studies confirm the value of the knowledge-enhanced, multi-step design, demonstrating suitability for lowresource, knowledge-centric applications.

Document