Info
We are very proud to announce the world's first and only Nvidia GH200 Grace-Hopper Superchip and Nvidia GB200 Grace-Blackwell Superchip-powered NV-linked, liquid-cooled, CDU-integrated, ready-to-use, "small-sized" rack server systems. Multiple NV-linked GH200 or GB200 act as a single giant GPU (CPU-GPU superchip) with one single giant coherent memory pool. We are the only ones who offer smaller systems than a complete rack with "only" 1, 2, 4, 8, 16 or 18 superchips (NVL2, NVL4, NVL8, NVL16 and NVL36). If you like AMD, we offer Mi300 systems too. All systems have a coolant distribution unit (CDU) integrated into the rack, are ready-to-use and are perfect for inferencing insanely huge LLMs, quick fine-tuning and training of LLMs, image and video generation and editing.
Example use case 1: Inferencing Llama-3.1 405B, Mistral Large 2 123B or Nvidia Nemotron 70BLlama-3.1 405B: https://llama.comMistral Large 2 123B: https://mistral.ai/news/mistral-large-2407/Nvidia Nemotron 70B: https://huggingface.co/nvidia/Llama-3.1-Nemotron-70B-Instruct-HFLlama-3.1 405B, Mistral Large 2 123B and Nvidia Nemotron 70B are the most powerful open-source models by far and even beat GPT-4omni and Claude 3.5 Sonnet. Llama-3.1 405B with 8-bit quantization needs at least 405GB of memory to swiftly run inference! Mistral Large 2 123B with 8-bit quantization needs at least 123GB of memory to swiftly run inference! Nvidia Nemotron 70B with 8-bit quantization needs at least 70GB of memory to swiftly run inference! Luckily, GH200 has a minimum of 576GB, GB200(A) a minimum of 864GB (768GB). With GH200 Mistral Large 2 123B and Nvidia Nemotron 70B can be run in VRAM only for ultra high inference speed (approx. 50 tokens/sec). With Mi300 and GB200 Blackwell this is also possible for Llama-3.1 405B. With GB200 Blackwell you can expect up to 1000 tokens/sec. If the model is bigger than VRAM you can only expect approx. 1-10 tokens/sec. 4-bit quantization seems to be the best trade-off between speed and accuracy, but is natively only supported by GB200 Blackwell.Example use case 2: Fine-tuning Llama-3 405B with PyTorch FSDP and Q-LoraTutorial: https://www.philschmid.de/fsdp-qlora-llama3Models need to be fine-tuned on your data to unlock the full potential of the model. But efficiently fine-tuning bigger models like Llama-3 405B remained a challenge until now. This blog post walks you through how to fine-tune Llama 3 using PyTorch FSDP and Q-Lora with the help of Hugging Face TRL, Transformers, peft & datasets. Fine-tuning big models within a reasonable time requires special and beefy hardware! Luckily, GH200, GB200 or Mi300 are ideal to complete this task extremely quickly.Example use case 3: Generating videos with Mochi1Download: https://github.com/genmoai/modelsDownload (reduced VRAM): https://github.com/victorchall/genmoai-smolMochi1 is democratizing efficient video production for all.Generating videos with Mochi1 requires special and beefy hardware! Luckily, GH200 and GB200 are ideal for this task.Example use case 4: Image generation with Flux.1Download: https://github.com/black-forest-labs/fluxFlux.1 is the best image generator at the moment. And it's uncensored, too. In high-speed inference, FLUX requires approximately 33GB of VRAM for maximum speed. For training the FLUX model, more than 40GB of VRAM is needed. Luckily, GH200 has a minimum of 96GB, GB200(A) a minimum of 288GB, Mi300A has a minimum of 512GB and Mi300X has a minimum of 1.5TB.Example use case 5: Image editing with Omnigen or Nvidia Add-itOmnigen: https://github.com/VectorSpaceLab/OmniGenNvidia Add-it: https://research.nvidia.com/labs/par/addit/Omnigen and Add-it are the most innovative and easy to use image editors at the moment. For maximum speed in high resolution image generation and editing beefier hardware than consumer graphics cards is needed. Luckily, GH200 and GB200 excel at this task.Example use case 6: Video editing with AutoVFXDownload: https://haoyuhsu.github.io/autovfx-website/AutoVFX is the most innovative and easy to use video editor at the moment. For maximum speed in high resolution video editing beefier hardware than consumer graphics cards is needed. Luckily, GH200 and GB200 excel at this task.Example use case 7: Creating a Large Language Model from scratchTutorial: https://www.pluralsight.com/resources/blog/data/how-build-large-language-modelImagine stepping into the world of language models as a painter stepping in front of a blank canvas. The canvas here is the vast potential of Natural Language Processing (NLP), and your paintbrush is the understanding of Large Language Models (LLMs). This article aims to guide you, new to NLP, in creating your first Large Language Model from scratch, focusing on the Transformer architecture and utilizing TensorFlow and Keras. Taining a LLM from scratch within a reasonable time requires special and extremely beefy hardware! Luckily, GH200, GB200 or Mi300 are ideal for this task.Why should you buy your own hardware?"You'll own nothing and you'll be happy?" No!!! Never should you bow to Satan and rent stuff that you can own. In other areas, renting stuff that you can own is very uncool and uncommon. Or would you prefer to rent "your" car instead of owning it? Most people prefer to own their car, because it's much cheaper, it's an asset that has value and it makes the owner proud and happy. The same is true for compute infrastructure.Even more so, because data and compute infrastructure are of great value and importance and are preferably kept on premises, not only for privacy reasons but also to keep control and mitigate risks. If somebody else has your data and your compute infrastructure you are in big trouble.Speed, latency and ease-of-use are also much better when you have direct physical access to your stuff.With respect to AI and specifically LLMs there is another very important aspect. The first thing big tech taught their closed-source LLMs was to be "politically correct" (lie) and implement guardrails, "safety" and censorship to such an extent that the usefulness of these LLMs is severely limited. Luckily, the (open-source) tools are out there to build and tune AI that is really intelligent and really useful. But first, you need your own hardware to run it on.
What are the main benefits of GH200 Grace-Hopper and GB200 Grace-Blackwell?They have enough memory to run, tune and train the biggest LLMs currently available.Their performance in every regard is almost unreal (up to 8520 times faster than x86).There are no alternative systems with the same amount of memory.Ideal for AI, especially inferencing, fine-tuning and training of LLMs.Multiple NV-linked GH200 or GB200 act as a single giant GPU.Optimized for memory-intensive AI and HPC performance.Ideal for HPC applications like, e.g. vector databases.Easily customizable, upgradable and repairable.Privacy and independence from cloud providers.Cheaper and much faster than cloud providers. They can be very quiet (with liquid-liquid CDU). Reliable and energy-efficient liquid cooling.Flexibility and the possibility of offline use.Gigantic amounts of coherent memory.They are very power-efficient.The lowest possible latency.They are beautiful.CUDA enabled.Run Linux.GB200 Blackwell
The coming Nvidia GB200 Grace-Blackwell superchip has truly amazing specs to show off. GPTrack.ai ready-to-use rack server systems with multiple NV-linked Nvidia GB200 Grace-Blackwell (up to 72) will arrive Q1 2025. Be one of the first in the world to get a GB200 rack system. Order now! What is the difference to alternative systems?
The main difference between GH200/GB200 and alternative systems is that with GH200/GB200, the GPU is connected to the CPU via a 900 GB/s NVLink vs. 128 GB/s PCIe gen5 used by traditional systems. Furthermore, multiple superchips can be connected via 900/1800 GB/s NVLink vs. orders of magnitude slower network connections used by traditional systems. Since these are the main bottlenecks, GH200/GB200's high-speed connections directly translate to much higher performance compared to traditional architectures. Also, multiple NV-linked GH200 or GB200 act as a single giant GPU (CPU-GPU superchip) with one single giant coherent memory pool.
What is the difference to server systems of competitors?Size: We focus on systems that are not bigger than one single rack. With GB200 that gives you more than an exaflop of compute. If that is really not enough for you, we are happy to make you a custom offer. But for many people, one complete rack is more than needed and too expensive. That is why we also offer smaller systems with only 1, 2, 4, 8, 16 or 18 superchips (NVL2, NVL4, NVL8, NVL16 and NVL36). We are, to our knowledge, the only ones in the world where you can get systems smaller than a complete GB200 NVL72 rack.In-rack CDU: Our rack server systems come standard with liquid cooling and a CDU integrated directly into the rack. You can choose between an air-liquid and liquid-liquid CDU.Ready-to-use: In contrast to other vendors, our systems come fully integrated and ready-to-use. Everything that is needed is included and tested. All you need to do is plug your system in to run it.Technical details of our GH200/GB200 rackserver systems (base configuration)Standard 19-inch or 21-inch OCP rackLiquid-cooledIn-rack CDU (air-liquid or liquid-liquid)Multiple Nvidia GH200 Grace-Hopper SuperchipsMultiple Nvidia GB200 Grace-Blackwell SuperchipsMultiple 72-core Nvidia Grace CPUsMultiple Nvidia Hopper H100 Tensor Core GPUs (on request)Multiple Nvidia Hopper H200 Tensor Core GPUs (on request)Multiple Nvidia Blackwell B100 Tensor Core GPUsUp to 72x 480GB of LPDDR5X memory with error-correction code (ECC)Up to 13.5TB of HBM3e memoryUp to 30TB of total fast-access memoryNVLink-C2C: 900 GB/s of bandwidthGH200: Programmable from 450W to 1000W TDP (CPU + GPU + memory)GB200: Programmable from 1200W to 2700W TDP (CPU + 2 GPU + memory)Up to 6x power shelveUp to 72x PCIe gen5 M.2 slots on boardUp to 288x PCIe gen5 drives (NVMe)Up to 108x FHFL PCIe Gen5 x163 years manufacturer's warrantyUp to 48U 600 x 2616 x 1200 mm (23.6 x 103 x 47.2")Up to 1500 kg (3300 lbs)Optional componentsNIC Nvidia Bluefield-3 400GbNIC Nvidia ConnectX-7 200GbNIC Intel 100GbUp to 72x 4TB M.2 SSDUp to 288x 8TB E1.S SSDUp to 288x 60TB 2.5" SSDStorage controllerRaid controllerOS preinstalledAnything possible on request
Need something different? We are happy to build custom systems to your liking.
Compute performance of one GH20067 teraFLOPS FP641 petaFLOPS TF322 petaFLOPS FP164 petaFLOPS FP8Compute performance of one GB20090 teraFLOPS FP645 petaFLOPS TF3210 petaFLOPS FP1620 petaFLOPS FP840 petaFLOPS FP4Benchmarkshttps://github.com/mag-/gpu_benchmarkPhoronix has so far benchmarked the Grace CPU. More is coming soon:
https://www.phoronix.com/review/nvidia-gh200-gptshop-benchmarkhttps://www.phoronix.com/review/nvidia-gh200-amd-threadripperhttps://www.phoronix.com/review/aarch64-64k-kernel-perfhttps://www.phoronix.com/review/nvidia-gh200-compilershttps://www.phoronix.com/review/nvidia-grace-epyc-turinWhite paper: Nvidia GH200 Grace-Hopper white paper
Trademark information: Nvidia is a trademark of Nvidia corporation. ARM is a trademark of Arm Holdings plc.Download
Here you can find various downloads concerning our GH200, GB200 and Mi300 systems: operating systems, firmware, drivers, software, manuals, white papers, spec sheets and so on. Everything you need to run your system and more.
White papers Nvidia GH200 Grace-Hopper white paper Nvidia GB200 Grace-Blackwell white paper Developing for Nvidia superchips
Spec sheetsGH200 624GB: Spec sheet GH200 624GB.pdfGH200 Giga 624GB: Spec sheet GH200 Giga 624GB.pdfGH200 NVL2 1.2TB: Spec sheet GH200 NVL2 1.2TB.pdfMi300A 512GB: Spec sheet Mi300A 512GB.pdfMi300X 1.5TB: Spec sheet Mi300X 1.5TB.pdfGB200 Blackwell 864GB: Spec sheet GB200 Blackwell 864GB.pdfGB200 NVL4 tray only: Spec sheet GB200 NVL4 tray only.pdfGB200 NVL4 1.7TB: Spec sheet GB200 NVL4 1.7TB.pdfGB200 NVL8 3.5TB: Spec sheet GB200 NVL8 3.5TB.pdfGB200 NVL16 7TB: Spec sheet GB200 NVL16 7TB.pdfGB200 NVL36 15TB: Spec sheet GB200 NVL36 15TB.pdfGB200 NVL72 30TB: Spec sheet GB200 NVL72 30TB.pdf
ManualsOfficial Nvidia GH200 Manual: https://docs.nvidia.com/grace/#grace-hopperOfficial Nvidia Grace Manual: https://docs.nvidia.com/grace/#grace-cpuOfficial Nvidia Grace getting started: https://docs.nvidia.com/grace/#getting-started-with-nvidia-graceGH200 624GB: Manual GH200 624GB.pdfGH200 Giga 624GB: Manual GH200 Giga 624GB.pdfGH200 NVL2 1.2TB: Manual GH200 NVL2 1.2TB.pdfMi300A: Manual Mi300A.pdfMi300X: Manual Mi300X.pdfGB200 NVL: Manual GB200 NVL.pdf
Operating systems for Nvidia systemsUbuntu Server for ARM: https://cdimage.ubuntu.com/releases/24.04/release/ubuntu-24.04.1-live-server-arm64+largemem.iso
Any other ARM linux distribution with kernel > 6.5 should work just fine. Using the newest 64k kernel is highly recommended.
Operating systems for AMD systemsUbuntu Server for x86: https://ubuntu.com/download/server
Any other x86 linux distribution with kernel > 6.8 should work just fine. Using the newest kernel is highly recommended.
DriversNvidia GH200 drivers: https://www.nvidia.com/Download/index.aspx?lang=en-us
Select product type "data center", product series "HGX-Series" and operating system "Linux aarch64".Aspeed drivers: https://aspeedtech.com/support_driver/Nvidia Bluefield-3 drivers: https://developer.nvidia.com/networking/doca#downloadsNvidia ConnectX-7 drivers: https://network.nvidia.com/products/ethernet-drivers/linux/mlnx_en/Intel E810-CQDA2 drivers: https://www.intel.com/content/www/us/en/download/19630/intel-network-adapter-driver-for-e810-series-devices-under-linux.html?wapkw=E810-CQDA2Graid SupremeRAID SR-1010 drivers: https://docs.graidtech.com/#linux-driver
FirmwareGH200 624GB: Firmware GH200 624GB.tarNvidia Bluefield-3 firmware: https://network.nvidia.com/support/firmware/bluefield3/Nvidia ConnectX-7 firmware: https://network.nvidia.com/support/firmware/connectx7/Intel E810-CQDA2 firmware: https://www.intel.com/content/www/us/en/search.html?ws=idsa-default#q=E810-CQDA2
Top open source LLMsLlama 3.1 and 3.2: https://www.llama.com/Mistral Large 2 123B: https://huggingface.co/mistralai/Mistral-Large-Instruct-2407Pixtral Large 123B: https://mistral.ai/news/pixtral-large/Nvidia NVLM-1.0-D-72B: https://huggingface.co/nvidia/NVLM-D-72BNvidia Llama-3.1 Nemotron 70B: https://huggingface.co/nvidia/Llama-3.1-Nemotron-70B-Instruct-HFLlama-3.2 Vision 90B: https://huggingface.co/meta-llama/Llama-3.2-90B-VisionLlama-3.1 405B: https://huggingface.co/meta-llama/Llama-3.1-405B
SoftwareNvidia Github: https://github.com/NVIDIANvidia CUDA: https://developer.nvidia.com/cuda-downloads?target_os=Linux&target_arch=arm64-sbsaNvidia Container-toolkit: https://github.com/NVIDIA/nvidia-container-toolkitNvidia Tensorflow: https://github.com/NVIDIA/tensorflowNvidia Pytorch: https://catalog.ngc.nvidia.com/orgs/nvidia/containers/pytorchNvidia NIM models: https://build.nvidia.com/explore/discoverNvidia Triton inference server: https://www.nvidia.com/de-de/ai-data-science/products/triton-inference-server/Nvidia NeMo Customizer: https://developer.nvidia.com/blog/fine-tune-and-align-llms-easily-with-nvidia-nemo-customizer/Huggingface open-source LLMs: https://huggingface.co/modelsHuggingface text generation inference: https://github.com/huggingface/text-generation-inferencevLLM - inference and serving engine: https://github.com/vllm-project/vllmvLLM docker image: https://hub.docker.com/r/drikster80/vllm-gh200-openaiOllama - run LLMs locally: https://ollama.com/Open WebUI: https://github.com/open-webui/open-webui/Fine-tune Llama 3 with PyTorch FSDP and Q-Lora: https://www.philschmid.de/fsdp-qlora-llama3/Perplexica: https://github.com/ItzCrazyKns/PerplexicaMorphic: https://github.com/miurla/morphicOpen-Sora: https://github.com/hpcaitech/Open-SoraFlux.1: https://github.com/black-forest-labs/fluxStorm: https://github.com/stanford-oval/stormStable Diffusion 3.5: https://huggingface.co/stabilityai/stable-diffusion-3.5-largeGenmo Mochi1: https://github.com/genmoai/modelsGenmo Mochi1 (reduced VRAM): https://github.com/victorchall/genmoai-smolRhymes AI Allegro: https://github.com/rhymes-ai/AllegroOmniGen: https://github.com/VectorSpaceLab/OmniGenSegment anything: https://github.com/facebookresearch/segment-anythingAutoVFX: https://haoyuhsu.github.io/autovfx-website/DimensionX: https://chenshuo20.github.io/DimensionX/Nvidia Add-it: https://research.nvidia.com/labs/par/addit/MagicQuill: https://magicquill.art/demo/
BenchmarkingGPU benchmark: https://github.com/mag-/gpu_benchmarkOllama benchmark: https://llm.aidatatools.com/results-linux.phpPhoronix test suite: https://www.phoronix-test-suite.com/MLCommons: https://mlcommons.org/benchmarks/Artifical Analysis: https://artificialanalysis.ai/Lmarena: https://lmarena.ai/Contact
Email: x@GPTrack.ai
GPTrack.ai UG (limited)
Sachsenhof 1
96106 Ebern
Germany
Trade register Bamberg HRB 11581
VAT: DE367543621
Lucid: DE5389117218256
WEEE: DE56766815
We accept the following currencies: AED, AUD, CAD, CHF, CNY, CZK, DKK, GBP, HKD, HUF, IDR, INR, JPY, MXN, MYR, NOK, NZD, PHP, PLN, QAR, RON, SEK, SGD, THB, TRY, USD, ZAR.
Payment is possible via wire transfer or credit card (3% fee).
Try
Try before you buy. You can apply for remote testing of a GH200, GB200 or Mi300 system. After approval, you will be given login credentials for remote access. If you want to come by and see it for yourself and run some tests, that is also possible any time.
Currently available for testing: GH200 624GB
Apply via email: x@GPTrack.ai