{"id":87940,"date":"2026-01-14T06:00:00","date_gmt":"2026-01-14T14:00:00","guid":{"rendered":"https:\/\/phisonblog.com\/?p=87940"},"modified":"2026-01-22T10:04:10","modified_gmt":"2026-01-22T18:04:10","slug":"phison-demonstrates-405b-parameter-llm-fine-tuning-with-aidaptiv-on-just-two-gpus","status":"publish","type":"post","link":"https:\/\/phisonblog.com\/zh-tw\/phison-demonstrates-405b-parameter-llm-fine-tuning-with-aidaptiv-on-just-two-gpus\/","title":{"rendered":"\u7fa4\u806f\u96fb\u5b50\u5229\u7528 aiDAPTIV+ \u5728\u50c5\u5169\u584a GPU \u4e0a\u5c55\u793a\u4e86 4050 \u5104\u53c3\u6578 LLM \u5fae\u8abf\u00a0"},"content":{"rendered":"

[et_pb_section fb_built=”1″ _builder_version=”4.16″ _module_preset=”default” custom_margin=”0px||||false|false” custom_padding=”0px||||false|false” locked=”off” global_colors_info=”{}”][et_pb_row _builder_version=”4.16″ _module_preset=”default” width=”100%” max_width=”100%” custom_margin=”||||false|false” custom_padding=”0px||||false|false” global_colors_info=”{}”][et_pb_column type=”4_4″ _builder_version=”4.16″ _module_preset=”default” global_colors_info=”{}”][et_pb_text _builder_version=”4.27.4″ _module_preset=”default” header_2_line_height=”1.7em” header_3_line_height=”1.7em” custom_margin=”||-10px||false|false” custom_padding=”||0px||false|false” locked=”off” global_colors_info=”{}”]<\/p>\n

At SC25, Phison showed off the potential of its aiDAPTIV+ hardware and software solution by fine-tune training the Llama 3.1 405 billion parameter model on a single server equipped with two GPUs and 192 GB of VRAM. This task normally requires a combined VRAM pool of over 7 TB and multiple NVIDIA servers \u2014 a large and costly setup. It was all done on a single system that would cost around $50,000, thanks to the power of aiDAPTIV+.<\/span>\u00a0<\/span><\/p>\n

The secret sauce in making this work was an 8TB aiDAPTIVCache SSD, used to store the calculated weights and values for the LLM. aiDAPTIV+ breaks down the complex task of fine-tuning into smaller, more manageable chunks rather than trying to fit everything into VRAM. The result is the democratization of access to more powerful AI models for a wide range of organizations.<\/span>\u00a0<\/span><\/p>\n

As an example of the versatility that aiDAPTIV+ brings to LLM fine-tune training, this system completed a task that would normally require a dozen NVIDIA HGX H100 servers, each with eight H100 GPUs. These servers require far more space and electricity, and costs are significantly higher than the aiDAPTIV+ server that accomplished the same task inside the Phison booth.<\/span>\u00a0<\/span><\/p>\n

 <\/p>\n

\"\"<\/a>
View: Phison aiDAPTIV+ Solution<\/div><\/div><\/div><\/div><\/div>\n

\u00a0<\/h3>\n

What is aiDAPTIV+?<\/span>\u00a0<\/span><\/h3>\n

aiDAPTIV+ is a hardware and software solution engineered to extend the memory space for fine-tune training of AI models and accelerate time to first token (TTFT) for inference workloads. A specialized aiDAPTIVCache SSD serves as a staging area for the unpacked LLM\u2019s tensor weights and vectors for training. The aiDAPTIVLink middleware manages memory allocation and determines how to best utilize available resources, shifting data to and from the aiDAPTIVCache SSD into VRAM or DRAM as needed.<\/span>\u00a0<\/span><\/p>\n

aiDAPTIVCache SSDs are purpose built for this task, with an extremely high endurance rating of 100 DWPD (drive writes per day). The SSDs achieve this level of endurance and availability of constant high-speed throughput utilizing SLC NAND. It\u2019s possible to write over 300TB per day to an aiDAPTIVCache SSD under heavy use, which could burn out regular SSDs in less than two months. aiDAPTIVCache drives are rated to handle this demanding scenario for five years of constant use.<\/span>\u00a0<\/span><\/p>\n

While the time to complete a fine-tune training process increases compared with utilizing pure GPU VRAM, most of the reduced performance comes from having fewer GPUs and less compute available. The aiDAPTIVCache SSDs are fast enough to not cause a severe bottleneck, resulting in training times around 5% longer than pure GPU-based training on a similar number of GPUs. Direct comparisons aren\u2019t particularly meaningful, however, as aiDAPTIV+ enables the use of significantly larger models on GPUs that otherwise can\u2019t fine-tune the dataset.<\/span><\/p>\n

 <\/p>\n

\"\"<\/p>\n

aiDAPTIV+ AITPC running Pro Suite on two RTX 4000 Ada GPUs.<\/span><\/i>\u00a0<\/span><\/p>\n

\u00a0<\/h3>\n

Llama 3.1 405B aiDAPTIV+ server training details<\/span>\u00a0<\/span><\/h3>\n

The server utilized by Phison at SC25 featured the following hardware that enabled training of the Llama 3.1 405B model.<\/span>\u00a0<\/span><\/p>\n

\"\"<\/p>\n

 <\/p>\n

aiDAPTIV+ supports a variety of modern platforms, including Intel and AMD CPUs, ARM processors, a variety of motherboards and chipsets, and a wide range of memory capacities. The CPU isn\u2019t a major factor in running aiDAPTIV+, but workstation and server platforms provide significantly higher connectivity options. Client solutions work but don\u2019t support as many PCIe lanes, which has a direct impact on the performance of the storage devices as well as the GPUs \u2014 especially when using more than one GPU and multiple SSDs.<\/span>\u00a0<\/span><\/p>\n

The Phison SC25 server used Intel\u2019s Xeon W5-3435X processor from the Sapphire Rapids family, which provides 112 total lanes of PCI Express 5.0 connectivity. Each of the RTX Pro 6000 GPUs uses a x16 PCIe Gen5 slot, with the SSDs using U.2 x4 connections. The server as configured represents more of a baseline configuration, with plenty of room for expansion options including additional aiDAPTIVCache SSDs and GPUs to increase performance.<\/span>\u00a0<\/span><\/p>\n

The aiDAPTIVCache SSD used at SC25 was an 8TB U.2 model, a required capacity to handle the training of a 405 billion parameter model. Fine-tune training of an LLM requires approximately 20 times the memory capacity as the number of parameters. In this case, 405 billion parameters equates to around 8 TB of memory. That memory is traditionally VRAM, or system RAM in some scenarios, both of which are very expensive in such quantities. aiDAPTIV+ augments the available memory with 8 TB of NAND flash storage.<\/span>\u00a0<\/span><\/p>\n

It requires around 26 of the latest NVIDIA B300 Ultra accelerators to train a 405B model using a traditional approach of keeping all the data in VRAM rather than with aiDAPTIV+, which means using four servers with eight GPUs each \u2014 and each individual GPU would cost as much as the entire aiDAPTIV+ server. Alternatively, it takes about 40 NVIDIA B200 GPUs spread over five servers, 53 H200 GPUs running in seven servers, or 93 H100 accelerators housed in a dozen HGX H100 servers. It would also require all the networking and interconnect hardware, with a total power use in the 40 kW to 80 kW range. The aiDAPTIV+ server only used about 1 kW and required no additional infrastructure.<\/span>\u00a0<\/span><\/p>\n

\u00a0<\/h3>\n

aiDAPTIV+ training results and performance<\/span>\u00a0<\/span><\/h3>\n

Phison used the Dolly creative writing benchmark found on Hugging Face to fine-tune the Llama 3.1 405B LLM for the SC25 demonstration. Different datasets affect the amount of time required for training, though scaling would still be similar across different hardware solutions.<\/span>\u00a0<\/span><\/p>\n

The base Llama-3.1-405B-Instruct model is 2 TB in size, using the Safetensors format. The first step of the training process consists of unpacking the LLM tensors into memory \u2014 or to the aiDAPTIVCache SSD in this case. This results in over 5 TB of data and requires about 30 minutes to generate, with peak requirements during the training process of over 7 TB of memory utilization.<\/span>\u00a0<\/span><\/p>\n

The Dolly dataset consists of 771K characters and 672 Q\/A pairs, and the aiDAPTIV+ server completed the fine-tune training at a rate of 25 hours and 50 minutes per epoch. The entire process finished in two days and four hours with a training run of two epochs.<\/span>\u00a0<\/span><\/p>\n

Over the four days the aiDAPTIV+ server ran at SC25, aiDAPTIV+ wrote hundreds of terabytes of data to the aiDAPTIVCache SSD. For typical enterprise SSDs rated at 1 DWPD, that level of writes will wear out the flash in a matter of weeks \u2014 less for lower capacity drives. With an endurance rating of 100 DWPD, Pascari aiDAPTIVCache SSDs are tailor made to handle this type of workload, running 24\/7\/365 for five years.<\/span>\u00a0<\/span><\/p>\n

 <\/p>\n

\"\"<\/p>\n

Pascari AI200E 8TB U.2 SSD used for fine-tune training of Llama 405B.<\/span><\/i>\u00a0<\/span><\/p>\n

\u00a0<\/h3>\n

Start doing more with aiDAPTIV+<\/span>\u00a0<\/span><\/h3>\n

aiDAPTIV+ provides an important path for expanding the available memory for fine-tune training, and with a global shortage of DRAM causing a spike in memory prices, it does so at a fraction of the cost. It opens up the potential to leverage AI at all levels of business, using on-premises hardware that doesn\u2019t require a massive financial investment, a data center, or large amounts of electricity. aiDAPTIV+ works for research and educational facilities, allowing experimentation with larger models without expensive cloud contracts or data sovereignty concerns.<\/span>\u00a0<\/span><\/p>\n

Phison\u2019s SC25 demonstration proves that a massive data center and costly infrastructure isn\u2019t needed to work with the biggest AI models. aiDAPTIV+ delivers a practical, affordable path to fine-tuning models once reserved for hyperscalers by combining purpose-built aiDAPTIVCache SSDs with intelligent memory orchestration. For organizations looking to unlock larger models and true on-premises AI independence, aiDAPTIV+ represents a compelling shift in what\u2019s possible. <\/span><\/p>\n

Visit Phison<\/span><\/a> to explore how aiDAPTIV+ can transform your AI workflows and see the technology in action.<\/span>\u00a0<\/span><\/p>\n

[\/et_pb_text][\/et_pb_column][\/et_pb_row][et_pb_row disabled_on=”on|on|on” _builder_version=”4.16″ _module_preset=”default” width=”100%” max_width=”100%” custom_margin=”||||false|false” custom_padding=”0px||||false|false” disabled=”on” saved_tabs=”all” global_colors_info=”{}”][et_pb_column type=”4_4″ _builder_version=”4.16″ _module_preset=”default” global_colors_info=”{}”][et_pb_text _builder_version=”4.27.4″ _module_preset=”default” global_colors_info=”{}”]<\/p>\n

Frequently Asked Questions (FAQ) :<\/strong><\/h3>\n

[\/et_pb_text][et_pb_toggle title=”How are universities driving the next wave of AI innovation?” _builder_version=”4.27.4″ _module_preset=”default” global_colors_info=”{}”]<\/p>\n

Universities are combining interdisciplinary research with AI from genomics to climate modeling and social sciences to solve global problems. Their open, collaborative environments accelerate model training and experimentation at a scale once reserved for <\/span>hyperscalers<\/span>.<\/span><\/p>\n

[\/et_pb_toggle][et_pb_toggle title=”Why is data throughput critical in academic AI research?” _builder_version=”4.27.4″ _module_preset=”default” global_colors_info=”{}”]<\/p>\n

AI models thrive on massive datasets. Without high-speed storage and efficient data movement, GPUs <\/span>remain<\/span> idle. Each millisecond of delay increases costs and slows breakthroughs in genomics, climate prediction, and behavioral studies.<\/span><\/p>\n

[\/et_pb_toggle][et_pb_toggle title=”What challenges do universities face when scaling AI infrastructure?” _builder_version=”4.27.4″ _module_preset=”default” global_colors_info=”{}”]<\/p>\n

Budgets, compliance, and legacy storage architectures often constrain performance. Many academic clusters have expanded GPU capacity but neglected the I\/O bottlenecks that prevent full <\/span>utilization<\/span> of compute power.<\/span><\/p>\n

[\/et_pb_toggle][et_pb_toggle title=”How does Phison\u2019s aiDAPTIV+ address these performance bottlenecks?” _builder_version=”4.27.4″ _module_preset=”default” global_colors_info=”{}”]<\/p>\n

aiDAPTIV<\/span>+ enhances throughput by offloading pre-processing and data management directly to <\/span>the SSD layer. This reduces <\/span>GPU<\/span> idle time, accelerates workloads, and maximizes return on existing hardware investments.<\/span><\/p>\n

[\/et_pb_toggle][et_pb_toggle title=”What types of research benefit most from Phison\u2019s technology?” _builder_version=”4.27.4″ _module_preset=”default” global_colors_info=”{}”]<\/p>\n

aiDAPTIV<\/span>+ delivers measurable improvements across <\/span>genomics<\/span> sequencing, high resolution climate simulations, and social data analytics, anywhere rapid access to small or unstructured files <\/span>determines<\/span> discovery speed.<\/span><\/p>\n

[\/et_pb_toggle][et_pb_toggle title=”Can universities maintain compliance while using aiDAPTIV+?” _builder_version=”4.27.4″ _module_preset=”default” global_colors_info=”{}”]<\/p>\n

Yes. <\/span>aiDAPTIV<\/span>+ is engineered for data sovereignty. Sensitive research data stays on premises or in private clouds, ensuring compliance with institutional and cross border data protection mandates.<\/span><\/p>\n

[\/et_pb_toggle][et_pb_toggle title=”How does Phison\u2019s SSD controller technology differ from traditional storage?” _builder_version=”4.27.4″ _module_preset=”default” global_colors_info=”{}”]<\/p>\n

Unlike standard SSDs that act as passive storage, <\/span>Phison\u2019s<\/span> controllers actively <\/span>optimize<\/span> I\/O flow through intelligent data pathing. This transforms storage into a <\/span>high performance<\/span> processing engine for AI workloads.<\/span><\/p>\n

[\/et_pb_toggle][et_pb_toggle title=”How does aiDAPTIV+ lower total cost of research computing?” _builder_version=”4.27.4″ _module_preset=”default” global_colors_info=”{}”]<\/p>\n

By increasing performance through smarter storage orchestration, universities achieve faster results without continuously expanding GPU fleets, reducing both capital and operational expenses.<\/span><\/span><\/p>\n

[\/et_pb_toggle][et_pb_toggle title=”What role will aiDAPTIV+ play in AI sustainability?” _builder_version=”4.27.4″ _module_preset=”default” global_colors_info=”{}”]<\/p>\n

aiDAPTIV<\/span>+ improves energy efficiency by minimizing GPU idle cycles and unnecessary data movement. This enables universities to advance AI research while lowering the carbon and energy footprint of their <\/span>compute<\/span> operations.<\/span><\/p>\n

[\/et_pb_toggle][et_pb_toggle title=”How can universities get started with Phison\u2019s aiDAPTIV+?” _builder_version=”4.27.4″ _module_preset=”default” global_colors_info=”{}”]<\/p>\n

Phison<\/span> offers <\/span>aiDAPTIV<\/span>+ as a turnkey platform that <\/span>integrates with<\/span> existing HPC and AI clusters. Universities can scale node counts as research grows, gaining hyperscale level performance within academic budgets.<\/span><\/span>\u00a0<\/span><\/p>\n

[\/et_pb_toggle][\/et_pb_column][\/et_pb_row][\/et_pb_section]<\/p>\n","protected":false},"excerpt":{"rendered":"

At SC25, Phison showed off the potential of its aiDAPTIV+ hardware and software solution by fine-tune training the Llama 3.1 405 billion parameter model on a single server equipped with two GPUs and 192 GB of VRAM. This task normally requires a combined VRAM pool of over 7 TB and multiple NVIDIA servers \u2014 a […]<\/p>\n","protected":false},"author":78,"featured_media":87963,"comment_status":"closed","ping_status":"closed","sticky":false,"template":"","format":"standard","meta":{"_acf_changed":false,"_et_pb_use_builder":"on","_et_pb_old_content":"","_et_gb_content_width":"","inline_featured_image":false,"footnotes":""},"categories":[120,23],"tags":[22],"class_list":["post-87940","post","type-post","status-publish","format-standard","has-post-thumbnail","hentry","category-ai","category-all-posts","tag-long-content"],"acf":[],"_links":{"self":[{"href":"https:\/\/phisonblog.com\/zh-tw\/wp-json\/wp\/v2\/posts\/87940","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/phisonblog.com\/zh-tw\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/phisonblog.com\/zh-tw\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/phisonblog.com\/zh-tw\/wp-json\/wp\/v2\/users\/78"}],"replies":[{"embeddable":true,"href":"https:\/\/phisonblog.com\/zh-tw\/wp-json\/wp\/v2\/comments?post=87940"}],"version-history":[{"count":17,"href":"https:\/\/phisonblog.com\/zh-tw\/wp-json\/wp\/v2\/posts\/87940\/revisions"}],"predecessor-version":[{"id":88077,"href":"https:\/\/phisonblog.com\/zh-tw\/wp-json\/wp\/v2\/posts\/87940\/revisions\/88077"}],"wp:featuredmedia":[{"embeddable":true,"href":"https:\/\/phisonblog.com\/zh-tw\/wp-json\/wp\/v2\/media\/87963"}],"wp:attachment":[{"href":"https:\/\/phisonblog.com\/zh-tw\/wp-json\/wp\/v2\/media?parent=87940"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/phisonblog.com\/zh-tw\/wp-json\/wp\/v2\/categories?post=87940"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/phisonblog.com\/zh-tw\/wp-json\/wp\/v2\/tags?post=87940"}],"curies":[{"name":"\u53ef\u6fd5\u6027\u7c89\u5291","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}