What large-model training needs from the hardware
Training a large language model is limited by three things: how much of the model and its training state fits in GPU memory, how fast GPUs exchange gradients and activations, and whether the capacity is there for the whole run. HGX B300 is built for all three.
- Memory. 288 GB of HBM3e per GPU and 2.3 TB per server keep more of the model, optimiser state and activations on the GPU, so less is sharded or recomputed.
- Bandwidth. 1.8 TB/s NVLink between the eight GPUs of a server, and a 1.6 Tb/s ConnectX-8-ready fabric between servers.
- Continuity. A reserved cluster is not shared and is not reclaimed mid-run. Capacity is held for the term of the contract.
How much GPU memory does a training run need?
A common rule of thumb for mixed-precision training with the Adam optimiser is about 16 bytes per parameter for weights, gradients and optimiser state, before activations. Sharding (for example FSDP or ZeRO) spreads that state across all GPUs in the cluster.
| Model size | Training state (≈16 B/param) | Share of an 8-server B300 cluster (18.4 TB) |
|---|---|---|
| 7B parameters | ≈ 112 GB | under 1% |
| 70B parameters | ≈ 1.1 TB | about 6% |
| 400B parameters | ≈ 6.4 TB | about 35% |
Approximate, before activations, which depend on sequence length, batch size and checkpointing. Your engineers' own profile of the run is the number to plan on; we size the cluster with you.
Why train in the EU
Training data is often the most sensitive data a company holds. On Sapience, it is processed and stored on systems in Bratislava, operated by a European company under European law, with no dependency on non-EU hyperscalers for compute, storage or networking. See security and data residency for the controls and their certification status.
How you run it
Clusters are delivered as bare metal: you install the operating system image, drivers, frameworks and scheduler you already use, with direct access to the GPUs. Managed Kubernetes and SLURM are in development on the same infrastructure; see the platform roadmap.
Training cluster questions.
Anything else: sales@sapienceai.eu or +421 233 329 562.
What is the smallest training cluster?
One cluster of 8 HGX B300 servers (64 GPUs) on a multi-year term. Larger runs use 24 or 32 servers (192 or 256 GPUs).
When can training start?
HGX B300 reservations are open now, with first deliveries scheduled from December 2026. The delivery date for your cluster is confirmed in the contract.
Can I use SLURM or Kubernetes?
Yes, installed by your team on the bare-metal nodes. Managed SLURM and Kubernetes from Sapience are in development.
Does training data leave the EU?
No. Customer data is processed and stored on Sapience systems in Bratislava, and Sapience does not replicate customer data outside the EU. Backup and replication arrangements are defined in the contract.
Plan a training run.
Describe the model, the data and the timeline. You get a cluster recommendation and a written quote.