The capacity of solid-state storage devices is soaring, with 32 terabytes now commonplace and 256-terabyte models expected to start shipping early next year, but all that bounty has also introduced a problem: The time required to test drives before putting them into operation has skyrocketed.

Now Sandisk Corp. has released to open source a technique that reduces preconditioning times by as much as 90%. The method, called Sandisk Pseudo-Random, collapses what was once a multi-phase process into a single, carefully constructed write pass.

Preconditioning is a standard procedure used by disk drive manufacturers and enterprises to ensure acceptable performance in real-world, steady-state use. Historically, the process has relied on a two-phase approach: a full sequential write across the drive, followed by extended periods of random writes that redistribute overprovisioned space within the device. The second phase forces the drive to use garbage collection and internal data movement, which slows performance but reflects worst-case operating conditions.

Choke point

The traditional approach worked well when SSDs were relatively small, but Sandisk engineers said it has begun to break down as capacities have surged. Drives larger than 32 terabytes can take days or even weeks to reach steady state, turning preconditioning into a major drag on development cycles, qualification timelines and customer testing.

“With every run-break-fix cycle you have to precondition,” said Steven Sprouse, a distinguished engineer in Sandisk’s enterprise SSD architecture group. “You might be running a half-hour test, but you have to spend two or three days preconditioning the drive.”

Instead of writing the drive sequentially and then hammering it with random I/O, the solution Sprouse developed writes data in overlapping regions that deliberately create “invalid” data across the physical media.

The technique exploits the way overprovisioning works inside an SSD. Drives contain more physical flash storage than they expose to users, with the extra capacity reserved for background tasks such as garbage collection. In steady-state operation, that overprovisioned space becomes unevenly distributed across physical blocks over time. SPRandom recreates the distribution mathematically by writing overlapping ranges of logical block addresses, thereby invalidating some data as newer data is written elsewhere.

“The overlap forces the drive to invalidate some of the data,” Sprouse explained. “That’s the equivalent of putting overprovisioning into those blocks.”

By controlling the amount of overlap at different points along the drive, SPRandom can approximate the same internal state that conventional random preconditioning eventually produces, but in a fraction of the time. According to Sandisk, a process that once took more than 160 hours on a 32-terabyte-class drive can now be completed in roughly six hours.

The impact is felt most acutely during testing and qualification, where long preconditioning cycles can stall engineering progress. Sandisk said engineers aim to keep run-break-fix loops under a single day so problems can be identified and resolved quickly. As capacities increased, that goal had become unattainable.

Broken bottleneck

“If something takes three days, you’re literally holding up the deployment of a fleet of drives,” said Jeff Fochtman, who leads product marketing for the flash products group at Sandisk.

Disk makers don’t typically precondition every drive before shipment, so customers often run their own preconditioning cycles before deployment. Measuring performance under pessimistic, worst-case conditions avoids surprises once drives are placed into production.

Sandisk chose to release SPRandom as open source and integrate it directly into the widely used Flexible I/O Tester benchmarking tool. The decision was driven by the company’s long involvement in industry ecosystems such as the Open Compute Project and the Storage Networking Industry Association, Fochtman said.

“OCP is meant for all ships to rise,” he said. “Because this is a generic solution that works across any drive type, it seemed appropriate to be part of the industry push forward versus a commercialized approach.”

Open-source licensing also allows innovation to flourish, Fochtman said. Early adopters are already exploring how to scale the method beyond single drives to entire racks and storage systems,

Sandisk executives said they also expect that open-sourcing will accelerate the adoption of ultra-high-capacity SSDs by removing a practical barrier to testing and deployment. As artificial intelligence, analytics and hyperscale workloads push storage capacities higher, they argue that faster, more realistic preconditioning benefits vendors and customers alike.

“If it’s easier to precondition and test larger drives,” Sprouse said, “it will will help us and the industry.”

Image: SiliconANGLE/Google Whisk