Popular with enterprise users, Milvus handles billion-fold, high-dimensional vector representations of unstructured data and simplifies deploying AI and MLOps at scale
Zilliz, whose founders created the Milvus open-source project, announced major contributions to the Milvus 2.1 release. The added functionality further bridges the gap between data pools, removes data silos, and offers performance and availability enhancements to address developers’ most common concerns. Milvus is one of the world’s most advanced vector databases, capable of managing massive quantities of both structured and unstructured data to accelerate the development of next-generation data fabric.
“When it comes to unstructured data, solutions offered by industry incumbents tend to be add-on capabilities or tools in a legacy database management system, whereas Milvus is designed around unstructured data from day one and is now offering more built-in capabilities to unlock more powerful and integrated data processing.”
A graduated-stage open-source project under the LF AI & Data Foundation, Milvus is built for scalable similarity search and used by a wide array of enterprises across industries. It embraces distributed architecture and can easily scale as data volumes and workloads increase. Highly scalable, reliable, and exceptionally fast, Milvus supports DML operations (adding, deleting, updating) and near real-time search of vectors on a trillion-byte scale.
Latest ITechnology News: Amdocs’ Vubiquity Expands Content Services Agreement with Oi
With this 2.1 update, Milvus sees significant improvement in its performance, reducing search latency on million-scale datasets to five milliseconds, while further simplifying deployment as well as ops workflow.
Bridging Gaps and Improving Performance
Machine learning is producing vast pools of scalar and vector data on a daily basis. With the introduction of more scalar data types, Milvus 2.1 is bridging this critical gap between data pools.
“Data silos can now be better integrated and linked, enabling businesses to unlock the full potential of their data,” said Milvus project maintainer Xiaofan James Luan, who also serves as the director of engineering at Zilliz. “When it comes to unstructured data, solutions offered by industry incumbents tend to be add-on capabilities or tools in a legacy database management system, whereas Milvus is designed around unstructured data from day one and is now offering more built-in capabilities to unlock more powerful and integrated data processing.”
Zilliz’s contributions to the 2.1 release specifically include:
- An overall performance boost including reduced latency; highly improved throughput for small-NQ application scenarios, such as reverse image search and intelligent chatbot; support of multiple memory replicas for small tables to increase throughput; and 2x increase in search performance.
- Improved scalar data processing that adds Varchar into supported data types and supports creating indexes for scalar data, taking hybrid search to a more intuitive level.
- itec
- A developer-friendly ecosystem in the making that includes more tutorials for building real-world applications, connecting Milvus with open-source vector data ETL framework Towhee; and that adds Feder, an open-source tool that helps Milvus users select the index best suited to their application scenario by visualizing the process of vector similarity search.
In addition to the integration and security features enumerated, Milvus will provide more functionalities essential to modern security mechanisms, including ACL (Access Control Lists) and advanced encryption methods.
Latest ITechnology News: AdvoLogix Powers Integration Between iManage and Salesforce Sales Cloud
Commitment to Open-Source Ecosystems
“As data infrastructure for unstructured data, Milvus is revolutionary because it processes vector embeddings and not just strings. In the future, Zilliz, the company founded by the creators of Milvus, seeks to build an ecosystem of solutions around Milvus, and some of the projects that will contribute to this have already surfaced, including Towhee, our open-source vector data ETL framework, and Feder, an interactive visualization tool for unstructured data. With Milvus 2.1 and the new demos, users can see how these products can come together to solve a series of problems that involve unstructured data,” added Luan.
Zilliz is committed to the developer community and will continue to contribute to open-source projects like Milvus. The company’s technology has broad applications spanning new drug discovery, computer vision, recommender engines, chatbots, and much more.
Latest ITechnology News: SimSpace Chooses The Nuvias Group as its First Distributor in the UK
[To share your insights with us, please write to sghosh@martechseries.com]
