Why Freeze the Encoder? A Controlled Study of End-to-End Sequence-Graph Learning for User Behavior

arXiv:2510.25126v4 Announce Type: replace-cross Abstract: Modeling user behavior couples two distinct signals: a per-user sequence of events and a persistent user-to-user relation graph. Standard pipelines preserve these native formats by combining them in two stages: a sequence encoder is pretrained, its per-user embeddings are frozen, and a graph model is trained on top to allow for cheap serving and cross-task reuse. We ask a simple question: does training the sequence and graph models end-to-end help, and if so, when? To isolate the impact of the training strategy, we hold a standard sequence-encoder-to-GNN pipeline fixed and vary only the optimization method; on Brightkite, a matched control that starts both pipelines from the same pretrained encoder and changes only whether it is updated attributes a 10.6-point MRR gain to end-to-end updating. Across relationship prediction and fraud detection on five real-world datasets, we find that the benefit of end-to-end training is conditional: it improves clearly over frozen two-stage pipelines on relationship prediction, where the relation graph appears to carry much of the signal, whereas gains shrink where the predictive signal is largely node-local (fraud detection), a regime in which a text-fed relational graph transformer achieves competitive results. Furthermore, recasting the data as a temporal graph yields inconsistent results compared to this native sequence-graph formulation. Ultimately, this work provides a controlled empirical account of when jointly training the two structures helps.