Tag: AWS

Posted on by

Storage technology explained: Flash vs HDD

The past 12 months saw flash storage nudge into areas from which it had hitherto been absent. In particular, this was because of the availability of denser – and therefore cheaper per-gigabyte (GB) – quad-level cell (QLC) flash storage into array markets and use cases that were once considered nearline.

Alongside this, we saw the price-per-GB of flash drop towards the level of spinning disk hard disk drives (HDDs) then rebound rapidly as memory manufacturers chased profitability. Meanwhile, the keenest of flash storage advocates predicted the demise of the hard drive and the imminent victory of the all-flash datacentre.

In this article, we define enterprise flash storage, look into its QLC and triple-level cell (TLC) variants, the benefits of non-volatile memory express (NVMe) flash, and examine the pros and cons of flash versus HDD in terms of cost, performance, flash in the cloud, and the likelihood (or otherwise) of the all-flash datacentre.

What is enterprise flash storage?

Enterprise flash storage refers to systems that comprise multiple flash drives housed in datacentre rack-mounted array form factor products.

In enterprise flash storage arrays, the capacity of many drives is aggregated, with access to storage media governed by controller hardware.

The controller is compute that powers the intelligence needed to handle input/output (I/O) from hosts to the storage, decision-making over allocation of data to media, but also in flash arrays to carry out maintenance tasks such as wear levelling, garbage collection, and so on.

Enterprise flash storage array capacities run from tens of terabytes (TB) to many petabytes (PB). As with HDD-based arrays, access to storage can be block (for performance-hungry database use cases, for example), file (for general use and unstructured data) or object (for unstructured data also).

What is QLC flash storage?

QLC is the latest generation of flash storage media. QLC stands for quad-level cell. That means that every cell in the flash chip can store four bits of data using 16 states.

That means it can store more data in the same space than TLC flash, which is also widely available. Previously widely available were single-level cell (SLC) flash and multi-level cell (MLC, meaning two states), but these have been largely superseded now.

At the start of 2024, most enterprise storage arrays are built with TLC drives for general-purpose and mission-critical use cases. But QLC has edged into the mainstream and gained traction for unstructured data workloads, in particular with key enterprise storage array makers adding QLC-based products in the past year or so.

As manufacturers increase the number of possible states per cell, storage density increases and the cost of storage per GB decreases. But, as storage density increases in terms of cell capacity, issues can arise that can limit the endurance of flash media.

What is NVMe flash?

Non-volatile memory express (NVMe) is a protocol developed especially for use with flash storage. Prior to NVMe, flash drives used transport protocols that originated during the HDD era, namely Serial Advanced Technology Attachment (SATA) and Serial-Attached SCSI (SAS). In fact, these are still in use and arrays that use drives with such connectivity (2.5in and 3.5in form factor) are sold by the big storage suppliers.

But NVMe is at the forefront now for flash drive performance. NVMe’s key innovation was to optimise queues and buffers for use with flash, which improved performance many times over.

As a follow-on, suppliers then developed ways of allowing NVMe connectivity across physically more distant connections across the datacentre. Such NVMe-over-fabrics technologies include the ability to carry NVMe via Ethernet, Infiniband, TCP, RDMA (ie, memory-to-memory connectivity) and more.

What is HDD?

Hard disk drives (HDDs) that rely on magnetic read/write heads and mechanically spinning disks have been around for decades, with flash a competitor that has emerged in the past 10 years or so.

As with flash, HDDs can be aggregated into datacentre rack-mounted array products and the capacity of multiple drives pooled for enterprise users. In fact, HDD-based arrays long preceded enterprise flash arrays and are still widely used.  

What’s the difference in performance between flash and HDD?

When we look at flash versus disk, the key thing that stands out is that flash is fast – many times faster than spinning disk HDD.

Flash drives offer lower latency, with access times down to low milliseconds, or even microseconds, compared with the multiple milliseconds of spinning disk, particularly for reads. That means enterprise flash can also offer vastly more input/output operations per second (IOPS) when aggregated into a storage array.

In throughput terms, flash offers gigabit-per-second (Gbps) rates four or five times quicker than HDD.

Such rapidity has been the key draw for enterprise flash storage and is a result of the lack of moving parts. With spinning platters, HDD is limited by physics in ways that solid-state storage is not.

In terms of capacities, HDD is available in up to around 22TB units. And while some flash drives have been marketed that run to 60-plus terabytes, they generally come in smaller sizes, but part of that is because of cost. 

What’s the cost difference between flash and HDD?

In terms of per-GB cost at drive level, flash costs more than spinning disk.

Flash prices spiked significantly in late 2023 and the early months of 2024 as manufacturers throttled back production in an effort to raise prices and achieve profitability.

Solid-state drive (SSD) prices per gigabyte reached an average of $0.095/GB by April 2024, which was a rise of 26.67% since autumn 2023.

But, flash drive prices then fell steadily over the first three quarters of 2024 to an average of $0.085 per gigabyte (GB) in September 2024.

In October 2023, flash had averaged $0.075/GB while HDD averaged $0.05/GB for SAS and $0.035/GB for SATA drives.

Average spinning disk (SAS and SATA) hard drive prices held steady during the six months to September 2024 at $0.039 per gigabyte. That figure was $0.041/GB in early April.

For a customer that planned to deploy 20TB of flash, based on those prices, it would have cost $1,500 in October 2023, $1,900 in April 2024, and $1,700 in September 2024. That compares to the equivalent for spinning disk of $850 in October 2023 and $780 in September 2024.

Will flash kill HDD? How much longer for HDD?

In particular, Pure Storage has declared HDDs will be dead by 2028, with its flash products the chief agent in the cull, and all owing to its ability to aggregate much more flash capacity on its proprietary modules than occurs on commodity flash drives.

With flash module sizes of up to 300TB by 2026 promised by Pure, it contends that spinning disk will be commercially unviable.

Meanwhile, companies such as Panasas, which specialises in storage for unstructured data, point to hyperscaler datacentres’ overwhelming use of spinning disk in ratios up to 90/10 against flash. Panasas argues that there’s still a five-times differential between the lowest-cost flash and HDD, and that for most, something like the hyperscaler solution is optimal.  

When can you use flash and HDD in the cloud?

Enterprise users can also specify flash storage and spinning disk in the cloud. It is more likely in most cases that cloud storage will be specified by performance and cost criteria, in which case the customer may never know what media underlies it.

But it is possible also to specify flash storage in the cloud and the three largest hyperscalers – Amazon Web Services (AWS), Microsoft Azure and Google Cloud Platform (GCP) – have solid-state storage options that mix cost, capacity and performance. 

The hyperscalers all offer flash storage to support compute with service levels based on capacity and IOPS per volume that range from general-purpose to premium levels aimed at specific workloads (eg, SQL, Oracle, SAP Hana) and environments (eg, Windows, Lustre, MacOS).

There are also options aimed at flash for file storage and flash storage from named suppliers, such as Azure’s NetApp Files.

What is the all-flash datacentre?

For about a decade, the idea of the all-flash datacentre has been discussed. The all-flash datacentre replaces HDD and other media such as tape with flash storage.

Driving it is the continued decrease in the cost of flash storage – as with QLC flash – but also the advantages of flash in terms of rapid access. The latter becomes more relevant as customers want to run analytics on bigger subsets of their data.

So, for example, where backups may previously have been held on nearline media such as slower HDDs, advocates of flash for such use cases point to the ability to run artificial intelligence (AI) on large customer datasets and to gain value therefrom.

Also, with backups as an example, the idea of being able to recover quickly from flash media in case of a ransomware attack is another use case touted by all-flash datacentre boosters. 

When will the all-flash datacentre arrive?

While enthusiastic suppliers of flash storage such as Pure talk down the obstacles to the all-flash datacentre, analysts point to the spread of (especially QLC) flash into secondary workloads but not necessarily all use cases, with spinning disk likely to retain its usefulness for some time for some datasets.

Meanwhile, HDD suppliers such as Toshiba say around 85% of all data is still on spinning disk. That fact, it says, is not likely to change rapidly, not least because the flash capacity to replace it doesn’t exist.

Source

Posted on by

AWS widening scope of MFA programme after early success

Amazon Web Services (AWS) is to widen the scope of a mandatory multi-factor authentication (MFA) programme it introduced earlier this year, after seeing strong uptake among customers and a slump in password-related phishing attacks.

The cloud giant made MFA compulsory for management account root users in the AWS Management Console beginning in May 2024, starting with its largest accounts. In June, it added support for FIDO2 passkeys as an MFA method to give users more options, and expanded the original requirement to include root users in standalone accounts, too.

According to AWS principal product manager of account protection Arynn Crow, over 750,000 root users have enabled MFA since April, with customer registration rates more than doubling since the addition of FIDO2 passkeys to the mix. She claimed the policy change had prevented “greater than 99%” of password-related attacks.

“At AWS, we’ve built our services with secure-by-design principles from day one, including features that set a high bar for our customers’ default security posture,” said Crow. “Strong authentication is a foundational component in overall account security, and the use of MFA is one of the simplest and most effective ways to help prevent unauthorised individuals from gaining access to systems or data.”

Based on this early success, AWS will now be expanding MFA requirements to member accounts in AWS organisations from Spring 2025.

“Customers who have not enabled central management of root access will be required to register MFA for their AWS Organizations member account root users in order to access the AWS Management Console,” said Crow.

“As with our previous expansions to management and standalone accounts, we will roll this change out gradually and notify individual customers who are required to take action in advance, to help customers adhere to the new requirements while minimising impact to their day-to-day operations.”

No more passwords anymore

On the back of its early successes with an MFA mandate, Crow said AWS was keen to do more to shore up security for its customers, and had recognised another opportunity to try to eliminate unnecessary passwords for good.

She said that on top of the run-of-the-mill security issues seen with standard passwords, attempting to secure password-based authentication was introducing too much operational overhead for AWS customers, especially those operating at scale or subject to regulatory requirements to rotate their credentials frequently.

As such, AWS has now launched a new capability to centrally manage root access for accounts managed in AWS Organizations, enabling them to cut down on the number of passwords they need to manage while still keeping control over the use of root principals.

Crow explained that customers can now turn on centralised root access with a quick configuration change – either in the identity and access management console or the AWS command line interface – and then remove the long-term credentials of member account root users.

“This will improve the security posture of our customers while simultaneously reducing their operational effort,” she concluded.

Source

Posted on by

Gartner Symposium: Why ServiceNow wants to be seen as the AI platform for business transformation

ServiceNow and Rimini Street have expanded their partnership to enable organisations to use older enterprise resource planning (ERP) systems as a starting point for enterprise artificial intelligence (AI) initiatives.

Using a single architecture and single data model from ServiceNow with Rimini Street’s third-party enterprise software support, the offering is being positioned as a way to enable organisations to innovate across procurement, finance, supply chain, HR, customer service and IT.

The partnership with Rimini Street represents a major step in the company’s strategy to evolve from case management to delivering AI-enabled workflow management that can coordinate and orchestrate systems of record in business, and also improve by learning from the data these systems hold.

During a presentation at the Gartner Symposium in Barcelona, Hartmut Mueller, chief transformation officer at ServiceNow, described the company’s platform as “connecting all systems of record and connecting [their] metrics to the value stream”.

In other words, if an AI can learn from the data held in traditional enterprise systems, it can figure out how to improve the business processes these applications power.

AI without new enterprise systems

Business and IT leaders need to justify the cost of new platform investments, and this is where the partnership with Rimini Street has been positioned.

Speaking to Computer Weekly during the Gartner conference, Eric Helmer, chief technology officer at Rimini Street, said that in the past, the chief information officer (CIO) role has traditionally been back-office. “People were expected to keep the lights on, run the business of IT and firefight,” he said. “The new role of the CIO is in the front office.” This implies their actions have a direct impact on the organisation’s bottom line.

“Today’s CIOs are expected to be the ideas people, and they are expected to figure out things like artificial intelligence and come up with revolutionary ideas that will be game-changers for the business,” he added.

However, as Gartner warned at the start of the annual European conference, it’s easy to lose a lot of money on misguided AI initiatives. A Gartner survey of 300 CIOs conducted in July found that 90% believe managing costs is limiting the value they can get from AI. Moreover, as every enterprise software provider AI-enables their products, Gartner expects a large proportion of IT budgets are likely to be spent on these products. 

But ServiceNow, in collaboration with Rimini Street, is trying to offer a different approach, which relies on the third-party support provider’s track record in keeping ERP systems from the major providers running, even after mainstream support has officially ended. These older systems of records remain useful, if not essential, data sources for machine learning to improve ServiceNow’s understanding of how the business operates.

Helmer said the partnership with ServiceNow helps IT departments make more use of existing enterprise IT assets. “If the value of an ERP system for AI is the data it contains, then the version of ERP you run becomes irrelevant because you already have the data,” he said.

While ServiceNow is often seen as a company specialising in case management, Helmer said it has a raft of tools focused on IT modernisation, which sit on top of existing IT assets. These, he said, can be used to modernise the user interface, automate workflows, and run AI and predictive analytics. “GenAI capabilities can be achieved in a matter of months, instead of years,” he claimed, since the ERP system does not have to be upgraded to the latest AI-capable version first.

Bill McDermott, chairman and chief executive officer at ServiceNow, described the enterprise systems deployed in businesses as 20th-century systems that hundred progress. “ServiceNow’s partnership with Rimini Street gives customers a more unified, intelligent platform to maximise their existing software investments, for faster paths to transformation,” he said.

The fact that ServiceNow can be applied to manage business processes just as it’s used in case management offers a way, according to Helmer, to route and automate workflows between various line-of-business applications, which means IT leaders do not need to go through the cost and disruption of an enterprise application modernisation programme to get the benefits of AI.

Switching software maintenance to third-party support has traditionally been regarded as something only a few brave IT leaders embark on.

Being a premier sponsor at the Gartner Symposium, alongside the likes of AWS, PWC and SAP among others, is perhaps an indication of where ServiceNow sees itself in the corporate IT landscape. If it’s successful at offering an AI platform for running and optimising business processes, there may well be a compelling reason not to upgrade to the latest AI-enabled enterprise system and rely, instead, on third-party support of an existing ERP system.

Source