Training

Training runs and the compute, data and interconnect they consume.

2 articles

Fine-tuning economics2

About Training

What is in the training section?

The compute, data and interconnect a training run consumes, and what that translates to in cost and elapsed time. Fine-tuning gets its own treatment: it is a different economic problem from pre-training and a far more common one, and the useful question is usually whether it beats prompting or retrieval for the same outcome.

Is any of this relevant if I am not training a foundation model?

Most of it, yes. The interconnect and cluster reports explain why a training-shaped workload behaves differently from a serving one on the same hardware, which matters as soon as you fine-tune anything at scale. The cost work is written so that a small run can be estimated from it rather than only a frontier one.