Video: PCIe Gen5 Is Almost Here – What You Need to Know

StorageReview recently sat down with Phison Chief Technology Officer Sebastien Jean to talk about PCIe Gen5. While the full discussion ranged from recent NAND developments to supply chain challenges to managing heat to the advantages of ZNS and developing PHY in-house, the following is an edited excerpt that focuses on the benefits of Gen5 SSD controllers and how Phison evolved as a company to become a major player in SSD performance.

StorageReview: Phison is more than simply a controller company, right?

Sebastien: Phison actually started off about 20 years ago as one of the first companies making USB drives, and we’ve grown ever since and expanded into pretty much every segment of the storage device market. This includes USB, SATA, or NVMe, and every sort of card format that exists: SD, Micro SD, and so on. We also make industrial devices, and we have solutions in medical, aerospace and automotive. But we started off as a small company that made controllers and what we call modules, which means fully functional SSDs.

Up until the E16, our customers were driving what the requirements were in the SSD, and we designed to a spec. But as we transitioned to the E16, we realized we can either produce me-too products or we can become a technology company. And that’s where the inflection point happened. We decided that as a company we wanted to be leaders. That was a conscious decision by our CEO and our president. They determined the way that we’re going to differentiate ourselves and continue to grow is by looking at the trends and then reaching for that high bar and trying to get there on time and at cost. It was a subtle change in mindset, but it set off waves throughout the whole company.

And now we’ve grown into a $1.6 billion technology company. So we don’t think of ourselves as a controller company anymore, and we don’t think of ourselves as an SSD company anymore. We think of ourselves as a company that focuses on having that next-generation, cutting-edge NAND storage technology. We still design to our customers’ specs, but we need to have the technology ready to help them get products out to market on time.

StorageReview: You get mentioned in the same breath sometimes as Samsung, Western Digital (WD), Intel—all these other big brands that everyone knows. And then little plucky Phison comes along with this E18 controller and drops an SSD on the market that is arguably one of the best right now for high-end client computing. I mean, that’s pretty strong. How did the E18 come about?

Sebastien: Yeah, but remember, we’re not small. Most of these large companies might have two or three engineering teams working on SSD. We have 20. We’re huge. It’s just that nobody knows who we are because we sell through technology partners.

Our first Gen4 product was the E16, which ran at about 5 GB/s. And that was a high-profile product that we developed in collaboration with AMD. We did a lot of onsite testing with them, and we pulled off a very short development cycle. Though it only achieved 5 GB/s, it was very successful at the time. It was the only client Gen4 on the market for about 18 months and the next faster drive only achieved 3.4 GB/s.

By the time we began developing the E18, we knew we had to do better. We took the feedback that we got about the E16 and looked at how other drives were performing. We listened to the reviewer community and looked for general trends. We also looked at our rankings in the various tests that people run. And of course, we do our own testing, too. So none of the feedback about performance and latency was a surprise, and we had those on our to-do list of improvements. One big one was to improve our low queue-depth handling. That required a big review of the entire front-end command processing pipeline to find every little idle delay and then squeeze it out. It was a big engineering effort to find those microsecond and picosecond delays and then redesign the hardware to eliminate them.

That’s essentially what allowed the E18 to happen. That, and we coupled it with basically the fastest NAND available at the time, which I think was Micron B27, Micron B47 or B48. That careful attention to detail allowed us to be really successful with the E18.

StorageReview: For your customers that are using a controller set like E18, where can they differentiate, outside of aesthetics like heatsinks and those sorts of things in client drive world?

Sebastien: We can do things like change the overprovisioning. There are knobs that you can turn on the SSD, which trade off capabilities to align with customer needs. For example, you can increase the overprovisioning or change the cache policy—that is how long you wait before flushing the cache.

Another option is how much residual data is kept in the cache to reduce write amplification and improve latency at the cost of reduced burst length. One strategy is to keep the last written 100 MB of data written to the drive, because the most recently written data is also the most likely to be invalidated. So, if you just keep it in the SLC cache and you don’t flush it to TLC or QLC, then you can potentially reduce write amp and improve sustained performance by increasing block invalidation.

Reducing write amplification increases the drive life. But if you get a burst of activity, you’ve pre-consumed 100 MB and so your burst of activity might hit the end of the cache sooner than another design might have. So there are a lot of tradeoffs that we can do on the drive, and it really comes down to what our customers want.

We can also optimize for completely different workloads, like we’ve started to develop a firmware that goes toward Gaming 2.0 and what comes after, for example, Microsoft Direct Storage. One of the workloads that you will see is something called 64K texture streaming, though that shouldn’t be confused with something like 4K TV, it’s actually just the file size. The natural granularity of texture files on most gaming cards is 64K, and so what happens is the games start pulling textures just in time instead of going through the CPU storage and DRAM stacks. The GPU will start treating the SSD like a great big cache. And that’s where Gen4 and Gen5 come in.

StorageReview: As a segue into Gen5, can you talk about how the E26 came about?

Sebastien: Initially our enterprise story was focused on up-leveling a high-end client controller. So the E12 became the E12 DC, for data center, which is the lowest tier of enterprise. This class of drive is for people that basically need steady-state performance and consistent latency, but don’t really care about any other enterprise features.

But starting about two years ago, we decided that we were going to jump in and put all of our energy into our next growth segment. So we designed what we were calling the X1 controller, which is a very high-end enterprise controller. It’s capable of Gen4x8, which means the entire data path is scaled to 14 GB/s. It is well-suited for EDSFF, for example. This form factor can support up to eight lanes, though today the focus is on Gen4x4. At the time we started the design, it wasn’t known which way the market would trend, so we aimed high with the X1.

That was the first dedicated enterprise ASIC for Phison. And that architecture has become the basis for all of our high-end controllers now. The upcoming E26 falls into that category. Its internal design is very similar to the X1, with the difference that it has a Gen5 interface on the front end, and on the back end it has eight NAND channels instead of 16.

When we were coming up with the E26 architecture, we asked ourselves, is there enough of a market for Gen5 if we focus only on enterprise? We concluded that the enterprise market is still emerging. We also looked at the client market and considered something based on the E18, but with Gen5 on it. This, too, was hard to justify, given the expected initial market size. So we decided to use the enterprise architecture, but turn off the features that don’t make sense for the client world and get the chip into as many segments as possible. And that’s where the E26 came from.

StorageReview: Let’s get into Gen5. What does the E26 product get you, from a Gen4 to Gen5, and what were some of the challenges to get there?

Sebastien: Let me start off by saying that Gen5 runs at the same speed as DDR4. I mean, DDR4 has a range of speeds, but if you look at the low end of DDR4, that’s Gen5. They’re roughly the same. And so this is the first time that you have an SSD that can interact with the CPU at the same speed or in the same range of speeds as DDR. Now, an SSD will never replace DDR because DDR has ultra-low latency, and it can be addressed in very tiny granularities. Whereas functionally, regardless of what the LBA size is, an SSD works on a 4K granularity and works better with larger commands or deep command queues.

But what Gen5 means both to the client and the enterprise space is that the SSD is quickly becoming akin to a level three or level four cache, depending on which architecture you’re looking at. It’s quite a few steps away from the CPU, but it’s still in that crazy speed range the CPU expects from DRAM. Whereas, you know, four years ago, SSDs were maybe 2 to 3 GB/s, and that’s assuming you were using an NVMe SSD. If you had a SATA SSD, then it was 600 MB/s. But now we’re talking about an SSD that runs at 14 GB/s.

By treating the SSD as a level four cache, the CPU architecture naturally expects to work with longer cache lines. And as the CPU migrates data from level four cache all the way up to level one, its cache lines get shorter and shorter until they align with DRAM granularity. This means you no longer need to put a ton of DRAM in a system to run AI and machine learning applications.

If you were to buy 8 TB of DDR—and to actually get that to fit into the DIMM slots that are available—it’s going to cost you $64,000. If you want an 8 TB SSD today, it costs about $1400. So that’s what you’re looking at. You can do a lot with 8 TB of DRAM. And if your calculation space really does require 8 TB of DRAM, then that’s what you need to put in your system. But in most cases, your active data set is quite a bit smaller and you’re using DRAM to avoid storage latency. If you can now fetch that data at almost DRAM speeds off of your level four cache, it’s now feasible to trade off $64,000 worth of DRAM for $1400 worth of SSD. And that’s only one SSD. Most enterprise systems can accommodate many SSDs. 24 is not uncommon, so you could have a petabyte of data that can be accessed at almost DRAM speeds. Then just load in the subset of data you need on the fly into a reasonably sized DRAM pool. This vastly increases the functional dataset an AI can work with through machine learning and other types of models.

If you treat the SSD like a level four cache, you don’t need to be encumbered by something like XL or Z-NAND. You can work with a conventional SSD that’s running at those Gen5 speeds and you reap a huge benefit in cost reduction. I believe the arrival of TLC and even QLC in this space will kind of hit that memory nirvana that people have been looking for, something that’s kind of like DRAM but much cheaper. And if the flow is primarily read-intensive, like many data analysis workloads are, then QLC will work quite well. Particularly at those high densities. Because there are so many NAND die that could be activated in parallel that you’re just not going to see the QLC latency and that’ll make the drive even cheaper. It won’t be $1400. It’ll be less than that.

StorageReview: How does Phison manage the duality between client SSD needs and concerns versus enterprise, because they’re so different? It’s hard to imagine a lot of client systems really needing Gen5 SSDs in the next 18 months.

Sebastien: The way I look at it is that at the very minimum, you’re going to get an improvement to your load times. They will be faster. So there’s a drop-in benefit right there. The rest of it will come to pass as the gaming, OS, and large-application technology companies—like, for example, Adobe or those other big ones—adapt their applications to use this speed.

It goes back to what I was saying earlier, that the SSD now becomes a DRAM assist module, like a level four cache. From an OS perspective, what that means is that you don’t have to load everything in the DRAM before you can start doing real work. So your boot time is going to be faster and everything is going to be shifting towards just-in-time loading.

The amount of DRAM in the system will likely come down, which helps reduce cost, helps reduce power, and will make your battery last longer. The three biggest consumers of power on a battery-powered device are your screen, your DDR, and your CPU. So reducing the amount of DRAM reduces the amount of heat, reduces the cooling, allows devices to get smaller, and allows batteries to last longer. And having everything loading just in time off a very snappy SSD means that your PC experience is going to be a lot more like the one on your phone.

And then the other expected benefit of Gen5 will come with game texture streaming. It’s probably a couple of years away. But it’s kind of like a “chicken or the egg” thing. The technology has to come before it can be designed into higher upstream products. And so that’s what we’re doing. Will an average consumer tomorrow need a Gen5 SSD? No, probably not. But would a super gamer want one? Absolutely.