Hongyan Chang and 12 coauthors introduced Watermark Basin Alignment, or WM-Basin, in a paper submitted Oct. 27. The proposed test queries a model through its API for signs it was trained on another model’s watermarked outputs. It does not require access to the model’s weights or logits.
A model provider can watermark the outputs of a “teacher” model in hopes of leaving a detectable trace in a “student” trained on them. Existing black-box detectors look for that trace in the student’s output tokens. Chang and colleagues write that token-level signals are not always detectable, including when the teacher outputs used for distillation are sampled at a low temperature.
WM-Basin takes a different approach. It maps text generated by the model being tested to gradient representations using a fixed, clean probe model. The authors say it then projects out ordinary task fine-tuning directions and checks whether the remaining signal aligns with the watermark basin induced by the watermark key.
In mathematical-reasoning experiments involving student models ranging from 135 million to 8.2 billion parameters, the authors report detecting 79% of distilled models at n-gram=1, compared with 29% for a token-level baseline, at a 0.1% false-positive rate. At n-gram=3, they report 45% versus 1.3% at the same false-positive rate. They also say the signal survives quantization, pruning and reinforcement-learning post-training. Those results describe the models in their experiments, not identified outside copies.
Comments