As someone who's done something similar (https://blog.lukesalamone.com/posts/creating-tiny-semantic-s...) the expensive part wasn't the training itself but the data curation and evaluation post-training. For this, getting a reasonable distribution of tool calls when the tool call can be anything isn't easy.
Once you have that, the model is small enough batch sizes are probably enormous and training can probably be done on a consumer-grade GPU in a week or less. Or even faster on a bigger GPU.
Once you have that, the model is small enough batch sizes are probably enormous and training can probably be done on a consumer-grade GPU in a week or less. Or even faster on a bigger GPU.