Update allows distributed Hadoop data processing framework to access small files, crucial for its ability to manage the current wave of real-time data-driven applications
Quobyte Inc., the leading developer of scale-out deploy-anywhere software-defined storage (SDS), announced the availability of its Hadoop Driver. Quobyte’s new native driver for Hadoop addresses the limitations of the Hadoop Distributed File System’s (HDFS) high-capacity design within the enterprise. The new native driver brings significant benefits in optimizing Hadoop clusters for a much wider range of applications and workloads, and true file system sharing across object storage and applications.
“Today’s analytics solutions allow enterprises to extract important insight from large volumes of data, but with the increasing prevalence of AI and machine learning in data analytics applications HDFS’s batch processing limitations have been exposed,” said Bjӧrn Kolbeck, CEO of Quobyte. “By deploying Quobyte’s native Hadoop/HDFS driver, enterprises can now seamlessly share large amounts of file data with high performance across Hadoop/analytics, machine learning, and any Linux or Windows application.”
Recommended ITech News: Claroty Launches Research Arm team82, Finds Critical Vulnerabilities in Cloud-Based ICS Management Platforms
Hadoop is a powerful framework for big data processing, distributed across large amounts of commodity hardware within the enterprise. It has proven critical in helping enterprises manage the exponential growth of data, but in today’s data-driven environment, it lacks the ability to manage the real-time requirements from the new breed of analytic applications. Engineers have been working hard, with limited success, to enable small file processing by using NFS connectors.
Quobyte now offers the ability to seamlessly integrate small file processing into the Hadoop Framework, without the need for a new file system, to deliver the speedy and scalable file I/O that is needed for modern analytic and real-time applications, while integrating well with container technology.
Recommended ITech News: Shift Technology Announces Shift Claims Automation on Salesforce AppExchange, The World’s Leading Enterprise Cloud Marketplace
Accelerated by the rapid evolution of analytic and real-time applications, Gartner predicts that by 2024, 50% of global storage capacity will be deployed as SDS, either on-premises or on the public cloud (up from less than 15% in 2020). With this announcement, Quobyte and Hadoop bring together the ability to run on any distributed commodity x86 hardware, which unlocks the benefits of Hadoop while removing the storage limitations of HDFS and NFS, ensuring these modern workloads can run efficiently on Hadoop clusters.
In addition, the new native driver removes security complexities by leveraging an alternative to HDFS that allows for encryption at both the storage and networking levels.
Recommended ITech News: Kasten by Veeam Announces Availability of Kasten K10 Data Management Platform for Kubernetes Application Backup and Mobility on Red Hat Marketplace
