Fine-tuned a distilbert with ml-intern today for the first time.
The procedure is really straightforward, almost unexpectedly so.
- Found a few datasets relevant to the task (prompt injection detection)
- Experimented a bit with few different models
- Did a sample training run to validate the params
- Did a full run afterwards
All via DeepSeek v4 Flash from OpenRouter for <$1. Working on an HRM/recursive version now.
https://huggingface.co/av-codes/pi-detector-distilbert