What is the difference between Milvus and Weaviate? Using vector databases to power AI

Inquiry

No significant differences between the technologies were found. The Milvus Python client implements a search method that retrieves a list of vectors, which allows a multi-vector query. The Weaviate Python client only allows vector search for a single vector.

As in the case of the indexing time analysis, both engines show similar behavior when executing queries. Note that before explicitly loading script collections for a query, Milvus experienced a warm-up effect that significantly affected the average query execution time for this system. Even after eliminating this drawback, the system is still prone to uneven query execution times. Nevertheless, it has a clear advantage over Weaviate: the average query execution time is shorter for all scenarios considered. Below are histograms of the average query execution times for each technology.

Conclusions

In this study, we set out to compare popular open source Vector Similarity Engine (VSE) solutions that facilitate embedding search through approximated nearest high-dimensional vectors. These new frameworks provide a more efficient approach to storing vector data. However, they are relatively recent and have yet to finalize fundamental features such as horizontal scaling, pagination, interleaving, and GPU support.

We hope to host a product catalog of 300k to 5 million products and latent relationships between products, queries and items, as well as other customer experience data. To realize our plans, we need the VSE to support a large dataset with multiple vector representations and provide multi-access efficiency. Therefore, as part of this study, it is critical for us to ensure that the following criteria are met:

• VSE provides high quality (accuracy) results;
• This is made possible by low-level implementations of executive index types.
• VSE indexes the embeddings with satisfactory responsiveness;
• This is made possible by low-level implementations of executive index types.
• VSE fulfills requests at a high rate of speed;
• This is made possible by low-level implementations of executive index types.
• VSE enables horizontal scaling with load balancing and data redundancy to protect against hardware failures while increasing service capacity;
• VSE provides a collection of K best results, but the system that integrates it can iterate the K best results over a window of size N.

This experiment focused mainly on two engines: Milvus and Weaviate. An analysis of the quality of the results obtained was not performed within the scope of this study, as this would have required an additional study of the configuration:

• The embedding model(s) used to encode the information;
• index type used.

Given that the experimental setup depended on the same configuration of all engines under study, we fixed the indexing algorithm on HNSW. Currently, to understand the impact of the indexing algorithm on speed, search quality and memory, you can refer to this blog article.

When analyzing indexing and query times, Milvus consistently outperforms Weaviate, with indexing times being particularly notable for the S9 scenario.

These technologies imposed technical limitations, such as the lack of support for multiple encodings (which actually required modifications to adapt the S4 - S9 scenarios). However, in [1], multi-vector querying refers to query retrieval based on a list of vectors and indexing entities that have more than one representation. Additional information on the toolkit roadmap indicates that these (and other) features should be expected in the next stable release.