GASP: Graph-Augmented Soft Prompt Tuning for Few-Shot Text Classification

Abstract

Few-shot short text classification (STC) is challenging due to semantic sparsity and limited labeled supervision. Existing STC methods often alleviate sparsity by exploiting corpus-level structure, for example, through document/word graphs or additional objectives such as contrastive learning; however, these designs can increase pipeline complexity and training overhead. Soft prompt tuning is a lightweight and parameter-efficient alternative, yet it is typically sequence-based and does not explicitly leverage word co-occurrence structure. We propose GRAPHPROMPT, a soft prompt tuning method that incorporates a word co-occurrence graph by using a WORDGCN module to compute graph-informed representations, transforming them into continuous prompt embeddings, and prepending them to the input of a BERT classifier. Compared with graph-based STC methods, GRAPHPROMPT does not require document-level graphs or multi-stage training objectives; compared to vanilla soft prompts, it injects graph-derived information into the prompt embeddings. Experiments on six datasets under a unified 20-shot protocol (20 training and 20 validation instances per class) over five random splits show that GRAPHPROMPT consistently outperforms a graph-only WORDGCN baseline and standard soft prompt tuning, and yields additional gains over strong BERT-only tuning on sparse domains, while incurring only modest overhead relative to vanilla soft prompts.

[PDF]