Vector Podcast

Video: https://youtu.be/dVIPBxHJ1kQ 00:00 Intro 00:15 Greets for Sonam 01:02 Importance of metric learning 3:37 Sonam's background: Rasa, Qdrant 4:31 What's EmbedAnything 5:52 What a user gets 8:48 Do I need to know Rust? 10:18 Call-out to the community 10:35 Multimodality 12:32 How to evaluate quality of LLM-based systems 16:38 QA for multimodal use cases 18:17 Place for a human in the LLM craze 19:00 Use cases for EmbedAnything 20:54 Closing theme (a longer one - enjoy!) Show notes: - GitHub: https://github.com/StarlightSearch/EmbedAnything - HuggingFace Candle: https://github.com/huggingface/candle - Sonam's talk on Berlin Buzzwords 2024: https://www.youtube.com/watch?v=YfR3kuSo-XQ - Removing GIL from Python: https://peps.python.org/pep-0703 - Blind pairs in CLIP: https://arxiv.org/abs/2401.06209 - Dark matter of intelligence: https://ai.meta.com/blog/self-supervised-learning-the-dark-matter-of-intelligence/ - Rasa chatbots: https://github.com/RasaHQ/rasa - Prometheus: https://github.com/prometheus-eval/prometheus-eval - Dino: https://github.com/facebookresearch/dino

00:00 Intro 00:30 Greets for Doug 01:46 Apache Solr and stuff 03:08 Hello LTR project 04:42 Secret sauce of Doug's continuous blogging 08:50 SearchArray 13:22 Running complex ML experiments 17:29 Efficient search orgs 22:58 Writing a book on search and AI Show notes: - Doug's talk on Learning To Rank at Reddit delivered at the Berlin Buzzwords 2024 conference: https://www.youtube.com/watch?v=gUtF1gyHsSM - Hello LTR: https://github.com/o19s/hello-ltr - Lexical search for pandas with SearchArray: https://github.com/softwaredoug/searcharray - https://softwaredoug.com/ - What AI Engineers Should Know about Search: https://softwaredoug.com/blog/2024/06/25/what-ai-engineers-need-to-know-search - AI Powered Search: https://www.manning.com/books/ai-powered-search - Quepid: https://github.com/o19s/quepid - Branching out in your ML / search experiments: https://dvc.org/doc/use-cases - Doug on Twitter: https://x.com/softwaredoug - Doug on LinkedIn: https://www.linkedin.com/in/softwaredoug/

00:00 Intro 00:21 Guest Introduction: Eric Pugh 03:00 Eric's story in search and the evolution of search technology 7:27 Quepid: Improving Search Relevancy 10:08 When to use Quepid 14:53 Flash back to Apache Solr 1.4 and the book (of which Eric is one author) 17:49 Quepid Demo and Future Enhancements 23:57 Real-Time Query Doc Pairs with WebSockets 24:16 Integrating Quepid with Search Engines 25:57 Introducing LLM-Based Judgments 28:05 Scaling Up Judgments with AI 28:48 Data Science Notebooks in Quepid 33:23 Custom Scoring in Quepid 39:23 API and Developer Tools 42:17 The Future of Search and Personal Reflections Show notes: - Hosted Quepid: https://app.quepid.com/ - Ragas: Evaluation framework for your Retrieval Augmented Generation (RAG) pipelines https://github.com/explodinggradients... - Why Quepid: https://quepid.com/why-quepid/ - Quepid on Github: https://github.com/o19s/quepid

00:00 Intro 01:54 Reflection on the past year in AI 08:08 Reader LLM (and RAG) 12:36 Does it need fine-tuning to a domain? 14:20 How LLMs can lie 17:32 What if data isn't perfect 21:21 SWIRL's secret sauce with Reader LLM 23:55 Feedback loop 26:14 Some surprising client perspective 31:17 How Gen AI can change communication interfaces 34:11 Call-out to the Community

00:00 Intro 00:42 Louis's background 05:39 From Facebook to Rockset 07:41 Embeddings prior to deep learning / LLM era 12:35 What's Rockset as a product 15:27 Use cases 18:04 RocksDB as part of Rockset 20:33 AI capabilities: ANN index, hybrid search 25:11 Types of hybrid search 28:05 Can one learn the alpha? 30:03 Louis's prediction of the future of vector search 33:55 RAG and other AI capabilities 41:46 Call out to the Vector Search community 46:16 Vector Databases vs Databases 49:16 Question of WHY

Topics: 00:00 Intro - how do you like our new design? 00:52 Greets 01:55 Saurabh's background 03:04 Resume Matcher: 4.5K stars, 800 community members, 1.5K forks 04:11 How did you grow the project? 05:42 Target audience and how to use Resume Matcher 09:00 How did you attract so many contributors? 12:47 Architecture aspects 15:10 Cloud or not 16:12 Challenges in maintaining OS projects 17:56 Developer marketing with Swirl AI Connect 21:13 What you (listener) can help with 22:52 What drives you? Show notes: - Resume Matcher: https://github.com/srbhr/Resume-Matcher website: https://resumematcher.fyi/ - Ultimate CV by Martin John Yate: https://www.amazon.com/Ultimate-CV-Cr... - fastembed: https://github.com/qdrant/fastembed - Swirl: https://github.com/swirlai/swirl-search

Topics: 00:00 Intro 00:22 Quick demo of SWIRL on the summary transcript of this episode 01:29 Sid’s background 08:50 Enterprise vs Federated search 17:48 How vector search covers for missing folksonomy in enterprise data 26:07 Relevancy from vector search standpoint 31:58 How ChatGPT improves programmer’s productivity 32:57 Demo! 45:23 Google PSE 53:10 Ideal user of SWIRL 57:22 Where SWIRL sits architecturally 1:01:46 How to evolve SWIRL with domain expertise 1:04:59 Reasons to go open source 1:10:54 How SWIRL and Sid interact with ChatGPT 1:23:22 The magical question of WHY 1:27:58 Sid’s announcements to the community YouTube version: https://www.youtube.com/watch?v=vhQ5LM5pK_Y Design by Saurabh Rai: https://twitter.com/_srbhr_ Check out his Resume Matcher project: https://www.resumematcher.fyi/

Topics: 00:00 Intro 02:20 Atita’s path into search engineering 09:00 When it’s time to contribute to open source 12:08 Taking management role vs software development 14:36 Knowing what you like (and coming up with a Solr course) 19:16 Read the source code (and cook) 23:32 Open Bistro Innovations Lab and moving to Germany 26:04 Affinity to Search world and working as a Search Relevance Consultant 28:39 Bringing vector search to Chorus and Querqy 34:09 What Atita learnt from Eric Pugh’s approach to improving Quepid 36:53 Making vector search with Solr & Elasticsearch accessible through tooling and documentation 41:09 Demystifying data embedding for clients (and for Java based search engines) 43:10 Shifting away from generic to domain-specific in search+vector saga 46:06 Hybrid search: where it will be useful to combine keyword with semantic search 50:53 Choosing between new vector DBs and “old” keyword engines 58:35 Women of Search 1:14:03 Important (and friendly) People of Open Source 1:22:38 Reinforcement learning applied to our careers 1:26:57 The magical question of WHY 1:29:26 Announcements See show notes on YouTube: https://www.youtube.com/watch?v=BVM6TUSfn3E

Topics: 00:00 Intro 01:54 Things Connor learnt in the past year that changed his perception of Vector Search 02:42 Is search becoming conversational? 05:46 Connor asks Dmitry: How Large Language Models will change Search? 08:39 Vector Search Pyramid 09:53 Large models, data, Form vs Meaning and octopus underneath the ocean 13:25 Examples of getting help from ChatGPT and how it compares to web search today 18:32 Classical search engines with URLs for verification vs ChatGPT-style answers 20:15 Hybrid search: keywords + semantic retrieval 23:12 Connor asks Dmitry about his experience with sparse retrieval 28:08 SPLADE vectors 34:10 OOD-DiskANN: handling the out-of-distribution queries, and nuances of sparse vs dense indexing and search 39:54 Ways to debug a query case in dense retrieval (spoiler: it is a challenge!) 44:47 Intricacies of teaching ML models to understand your data and re-vectorization 49:23 Local IDF vs global IDF and how dense search can approach this issue 54:00 Realtime index 59:01 Natural language to SQL 1:04:47 Turning text into a causal DAG 1:10:41 Engineering and Research as two highly intelligent disciplines 1:18:34 Podcast search 1:25:24 Ref2Vec for recommender systems 1:29:48 Announcements For Show Notes, please check out the YouTube episode below. This episode on YouTube: https://www.youtube.com/watch?v=2Q-7taLZ374 Podcast design: Saurabh Rai: https://twitter.com/srvbhr

Toloka’s support for Academia: grants and educator partnerships https://toloka.ai/collaboration-with-educators-form https://toloka.ai/research-grants-form These are pages leading to them: https://toloka.ai/academy/education-partnerships https://toloka.ai/grants Topics: 00:00 Intro 01:25 Jenny’s path from graduating in ML to a Data Advocate role 07:50 What goes into the labeling process with Toloka 11:27 How to prepare data for labeling and design tasks 16:01 Jenny’s take on why Relevancy needs more data in addition to clicks in Search 18:23 Dmitry plays the Devil’s Advocate for a moment 22:41 Implicit signals vs user behavior and offline A/B testing 26:54 Dmitry goes back to advocating for good search practices 27:42 Flower search as a concrete example of labeling for relevancy 39:12 NDCG, ERR as ranking quality metrics 44:27 Cross-annotator agreement, perfect list for NDCG and Aggregations 47:17 On measuring and ensuring the quality of annotators with honeypots 54:48 Deep-dive into aggregations 59:55 Bias in data, SERP, labeling and A/B tests 1:16:10 Is unbiased data attainable? 1:23:20 Announcements This episode on YouTube: https://youtu.be/Xsw9vPFqGf4 Podcast design: Saurabh Rai: https://twitter.com/srvbhr

00:00 Introduction 01:11 Yaniv’s background and intro to Searchium & GSI 04:12 Ways to consume the APU acceleration for vector search 05:39 Power consumption dimension in vector search 7:40 Place of the platform in terms of applications, use cases and developer experience 12:06 Advantages of APU Vector Search Plugins for Elasticsearch and OpenSearch compared to their own implementations 17:54 Everyone needs to save: the economic profile of the APU solution 20:51 Features and ANN algorithms in the solution 24:23 Consumers most interested in dedicated hardware for vector search vs SaaS 27:08 Vector Database or a relevance oriented application? 33:51 Where to go with vector search? 42:38 How Vector Search fits into Search 48:58 Role of the human in the AI loop 58:05 The missing bit in the AI/ML/Search space 1:06:37 Magical WHY question 1:09:54 Announcements - Searchium vector search: https://searchium.ai/ - Dr. Avidan Akerib, founder behind the APU technology: https://www.linkedin.com/in/avidan-akerib-phd-bbb35b12/ - OpenSearch benchmark for performance tuning: https://betterprogramming.pub/tired-of-troubleshooting-idle-search-resources-use-opensearch-benchmark-for-performance-tuning-d4277c9f724 - APU KNN plugin for OpenSearch: https://towardsdatascience.com/bolster-opensearch-performance-with-5-simple-steps-ca7d21234f6b - Multilingual and Multimodal Search with Hardware Acceleration: https://blog.muves.io/multilingual-and-multimodal-vector-search-with-hardware-acceleration-2091a825de78 - Muves talk at Berlin Buzzwords, where we have utilized GSI APU: https://blog.muves.io/muves-at-berlin-buzzwords-2022-3150eef01c4 - Not All Vector Databases are made equal: https://towardsdatascience.com/milvus-pinecone-vespa-weaviate-vald-gsi-what-unites-these-buzz-words-and-what-makes-each-9c65a3bd0696 Episode on YouTube: https://youtu.be/EerdWRPuqd4 Podcast design: Saurabh Rai: https://twitter.com/srvbhr

Topics: 00:00 Intro 01:30 Doug’s story in Search 04:55 How Quepid came about 10:57 Relevance as product at Shopify: challenge, process, tools, evaluation 15:36 Search abandonment in Ecommerce 21:30 Rigor in A/B testing 23:53 Turn user intent and content meaning into tokens, not words into tokens 32:11 Use case for vector search in Maps. What about search in other domains? 38:05 Expanding on dense approaches 40:52 Sparse, dense, hybrid anyone? 48:18 Role of HNSW, scalability and new vector databases vs Elasticsearch / Solr dense search 52:12 Doug’s advice to vector database makers 58:19 Learning to Rank: how to start, how to collect data with active learning, what are the ML methods and a mindset 1:12:10 Blending search and recommendation 1:16:08 Search engineer role and key ingredients of managing search projects today 1:20:34 What does a Product Manager do on a Search team? 1:26:50 The magical question of WHY 1:29:08 Doug’s announcements Show notes: Doug’s course: https://www.getsphere.com/ml-engineering/ml-powered-search?source=Instructor-Other-070922-vector-pod Upcoming book: https://www.manning.com/books/ai-powered-search?aaid=1&abid=e47ada24&chan=aips Doug’s post in Shopify’s blog “Search at Shopify—Range in Data and Engineering is the Future”: https://shopify.engineering/search-at-shopify Doug’s own blog: https://softwaredoug.com/ Using Bayesian optimization for Elasticsearch relevance: https://www.youtube.com/watch?v=yDcYi-ANJwE&t=1s Hello LTR: https://github.com/o19s/hello-ltr Vector Databases: https://towardsdatascience.com/milvus-pinecone-vespa-weaviate-vald-gsi-what-unites-these-buzz-words-and-what-makes-each-9c65a3bd0696 Research: Search abandonment has a lasting impact on brand loyalty: https://cloud.google.com/blog/topics/retail/search-abandonment-impacts-retail-sales-brand-loyalty Quepid: https://quepid.com/ Podcast design: Saurabh Rai [https://twitter.com/srvbhr]

Topics: 00:00 Introduction 01:12 Malte’s background 07:58 NLP crossing paths with Search 11:20 Product discovery: early stage repetitive use cases pre-dating Haystack 16:25 Acyclic directed graph for modeling a complex search pipeline 18:22 Early integrations with Vector Databases 20:09 Aha!-use case in Haystack 23:23 Capabilities of Haystack today 30:11 Deepset Cloud: end-to-end deployment, experiment tracking, observability, evaluation, debugging and communicating with stakeholders 39:00 Examples of value for the end-users of Deepset Cloud 46:00 Success metrics 50:35 Where Haystack is taking us beyond MLOps for search experimentation 57:13 Haystack as a smart assistant to guide experiments 1:02:49 Multimodality 1:05:53 Future of the Vector Search / NLP field: large language models 1:15:13 Incorporating knowledge into Language Models & an Open NLP Meetup on this topic 1:16:25 The magical question of WHY 1:23:47 Announcements from Malte Show notes: - Haystack: https://github.com/deepset-ai/haystack/ - Deepset Cloud: https://www.deepset.ai/deepset-cloud - Tutorial: Build Your First QA System: https://haystack.deepset.ai/tutorials/v0.5.0/first-qa-system - Open NLP Meetup on Sep 29th (Nils Reimers talking about “Incorporating New Knowledge Into LMs”): https://www.meetup.com/open-nlp-meetup/events/287159377/ - Atlas Paper (Few shot learning with retrieval augmented large language models): https://arxiv.org/abs/2208.03299 - Tweet from Patrick Lewis: https://twitter.com/PSH_Lewis/status/1556642671569125378 - Zero click search: https://www.searchmetrics.com/glossary/zero-click-searches/ Very large LMs: - 540B PaLM by Google: https://lnkd.in/eajsjCMr - 11B Atlas by Meta: https://lnkd.in/eENzNkrG - 20B AlexaTM by Amazon: https://lnkd.in/eyBaZDTy - Players in Vector Search: https://www.youtube.com/watch?v=8IOpgmXf5r8 https://dmitry-kan.medium.com/players-in-vector-search-video-2fd390d00d6 - Click Residual: A Query Success Metric: https://observer.wunderwood.org/2022/08/08/click-residual-a-query-success-metric/ - Tutorials and papers around incorporating Knowledge into Language Models: https://cs.stanford.edu/people/cgzhu/ Podcast design: Saurabh Rai https://twitter.com/srvbhr

00:00 Introduction 01:10 Max's deep experience in search and how he transitioned from structured data 08:28 Query-term dependence problem and Max's perception of the Vector Search field 12:46 Is vector search a solution looking for a problem? 20:16 How to move embeddings computation from GPU to CPU and retain GPU latency? 27:51 Plug-in neural model into Java? Example with a Hugging Face model 33:02 Web-server Mighty and its philosophy 35:33 How Mighty compares to in-DB embedding layer, like Weavite or Vespa 39:40 The importance of fault-tolerance in search backends 43:31 Unit economics of Mighty 50:18 Mighty distribution and supported operating systems 54:57 The secret sauce behind Mighty's insane fast-ness 59:48 What a customer is paying for when buying Mighty 1:01:45 How will Max track the usage of Mighty: is it commercial or research use? 1:04:39 Role of Open Source Community to grow business 1:10:58 Max's vision for Mighty connectors to popular vector databases 1:18:09 What tooling is missing beyond Mighty in vector search pipelines 1:22:34 Fine-tuning models, metric learning and Max's call for partnerships 1:26:37 MLOps perspective of neural pipelines and Mighty's role in it 1:30:04 Mighty vs AWS Inferentia vs Hugging Face Infinity 1:35:50 What's left in ML for those who are not into Python 1:40:50 The philosophical (and magical) question of WHY 1:48:15 Announcements from Max 25% discount for the first year of using Mighty in your great product / project with promo code VECTOR: https://bit.ly/3QekTWE Show notes: - Max's blog about BERT and search relevance: https://opensourceconnections.com/blog/2019/11/05/understanding-bert-and-search-relevance/ - Case study and unit economics of Mighty: https://max.io/blog/encoding-the-federal-register.html - Not All Vector Databases Are Made Equal: https://towardsdatascience.com/milvus-pinecone-vespa-weaviate-vald-gsi-what-unites-these-buzz-words-and-what-makes-each-9c65a3bd0696 Watch on YouTube: https://youtu.be/LnF4hbl1cE4

Vector Podcast Live Topics: 00:00 Kick-off introducing co:rise study platform 03:03 Grant’s background 04:58 Principle of 3 C’s in the life of a CTO: Code, Conferences and Customers 07:16 Principle of 3 C’s in the Search Engine development: Content, Collaboration and Context 11:51 Balance between manual tuning in pursuit to learn and Machine Learning 15:42 How to nurture intuition in building search engine algorithms 18:51 How to change the approach of organizations to true experimentation 23:17 Where should one start in approaching the data (like click logs) for developing a search engine 29:36 How to measure the success of your search engine 33:50 The role of manual query rating to improve search result relevancy 36:56 What are the available datasets, tools and algorithms, that allow us to build a search engine? 41:56 Vector search and its role in broad search engine development and how the profession is shaping up 49:01 The magical question of WHY: what motivates Grant to stay in the space 52:09 Announcement from Grant: course discount code DGSEARCH10 54:55 Questions from the audience Show notes: - Grant’s interview at Berlin Buzzwords 2016: https://www.youtube.com/watch?v=Y13gZM5EGdc - “BM25 is so Yesterday: Modern Techniques for Better Search”: https://www.youtube.com/watch?v=CRZfc9lj7Po - “Taming text” - book co-authored by Grant: https://www.manning.com/books/taming-text - Search Fundamentals course - https://corise.com/course/search-fundamentals - Search with ML course - https://corise.com/course/search-with-machine-learning - Click Models for Web Search: https://github.com/markovi/PyClick - Trustworthy Online Controlled Experiments: A Practical Guide to A/B Testing, book by Ron Kohavi et al: https://www.amazon.com/Trustworthy-Online-Controlled-Experiments-Practical-ebook/dp/B0845Y3DJV - Quepid, open source tool and free service for query rating and relevancy tuning: https://quepid.com/ - Grant’s talk in 2013 where he discussed the need of a vector field in Lucene and Solr: https://www.youtube.com/watch?v=dCCqauwMWFE - CLIP model for multimodal search: https://openai.com/blog/clip/ - Demo of multimodal search with CLIP: https://blog.muves.io/multilingual-and-multimodal-vector-search-with-hardware-acceleration-2091a825de78 - Learning to Boost: https://www.youtube.com/watch?v=af1dyamySCs - Dmitry’s Medium List on Vector Search: https://medium.com/@dmitry-kan/list/vector-search-e9b564d14274

Topics: 00:00 Kick-off by Judy Zhu 01:33 Introduction by Dmitry Kan and his bio! 03:03 Daniel’s background 04:46 “Science is the difference between instinct and strategy” 07:41 Search as a personal learning experience 11:53 Why do we need Machine Learning in Search, or can we use manually curated features? 16:47 Swimming up-stream from relevancy: query / content understanding and where to start? 23:49 Rule-based vs Machine Learning approaches to Query Understanding: Pareto principle 29:05 How content understanding can significantly improve your search engine experience 32:02 Available datasets, tools and algorithms to train models for content understanding 38:20 Daniel’s take on the role of vector search in modern search engine design as the path to language of users 45:17 Mystical question of WHY: what drives Daniel in the search space today 49:50 Announcements from Daniel 51:15 Questions from the audience Show notes: [What is Content Understanding?. Content understanding is the foundation… | by Daniel Tunkelang | Content Understanding | Medium](https://medium.com/content-understanding/what-is-content-understanding-4da20e925974) [Query Understanding: An Introduction | by Daniel Tunkelang | Query Understanding](https://queryunderstanding.com/introduction-c98740502103) Science as Strategy [YouTube](https://www.youtube.com/watch?v=dftt6Yqgnuw) Search Fundamentals course - https://corise.com/course/search-fundamentals Search with ML course - https://corise.com/course/search-with-machine-learning Books: Faceted Search, by Daniel Tunkelang: https://www.amazon.com/Synthesis-Lectures-Information-Concepts-Retrieval/dp/1598299999 Modern Information Retrieval: The Concepts and Technology Behind Search, by Ricardo Baeza-Yates: https://www.amazon.com/Modern-Information-Retrieval-Concepts-Technology/dp/0321416910/ref=sr11?qid=1653144684&refinements=p_27%3ARicardo+Baeza-Yates&s=books&sr=1-1 Introduction to Information Retrieval, by Chris Manning: https://www.amazon.com/Introduction-Information-Retrieval-Christopher-Manning/dp/0521865719/ref=sr1fkmr0_1?crid=2GIR19OTZ8QFJ&keywords=chris+manning+information+retrieval&qid=1653144967&s=books&sprefix=chris+manning+information+retrieval%2Cstripbooks-intl-ship%2C141&sr=1-1-fkmr0 Query Understanding for Search Engines, by Yi Chang and Hongbo Deng: https://www.amazon.com/Understanding-Search-Engines-Information-Retrieval/dp/3030583333

Topics: 00:00 Intro 01:03 Yusuf’s background 03:00 Multimodal search in tech and humans 08:53 CLIP: discovering hidden semantics 13:02 Where to start to apply metric learning in practice. AutoEncoder architecture included! 19:00 Unpacking it further: what is metric learning and the difference with deep metric learning? 28:50 How Deep Learning allowed us to transition from pixels to meaning in the images 32:05 Increasing efficiency: vector compression and quantization aspects 34:25 Yusuf gives a practical use-case with Conversational AI of where metric learning can prove to be useful. And tools! 40:59 A few words on how the podcast is made :) Yusuf’s explanation of how Gmail smart reply feature works internally 51:19 Metric learning helps us learn the best vector representation for the given task 52:16 Metric learning shines in data scarce regimes. Positive impact on the planet 58:30 Yusuf’s motivation to work in the space of vector search, Qdrant, deep learning and metric learning — the question of Why 1:05:02 Announcements from Yusuf - Join discussions at Discord: https://discord.qdrant.tech - Yusuf's Medium: https://medium.com/@yusufsarigoz and LinkedIn: https://www.linkedin.com/in/yusufsarigoz/ - GSOC 2022: TensorFlow Similarity - project led by Yusuf: https://docs.google.com/document/d/1fLDLwIhnwDUz3uUV8RyUZiOlmTN9Uzy5ZuvI8iDDFf8/edit#heading=h.zftd93u5hfnp - Dmitry's Twitter: https://twitter.com/DmitryKan Full Show Notes: https://www.youtube.com/watch?v=AU0O_6-EY6s

Topics: 00:00 Introduction 01:21 Jo Kristian’s background in Search / Recommendations since 2001 in Fast Search & Transfer (FAST) 03:16 Nice words about Trondheim 04:37 Role of NTNU in supplying search talent and having roots in FAST 05:33 History of Vespa from keyword search 09:00 Architecture of Vespa and programming language choice: C++ (content layer), Java (HTTP requests and search plugins) and Python (pyvespa) 13:45 How Python API enables evaluation of the latest ML models with Vespa and ONNX support 17:04 Tensor data structure in Vespa and its use cases 22:23 Multi-stage ranking pipeline use cases with Vespa 24:37 Optimizing your ranker for top 1. Bonus: cool search course mentioned! 30:18 Fascination of Query Understanding, ways to implement and its role in search UX 33:34 You need to have investment to get great results in search 35:30 Game-changing vector search in Vespa and impact of MS Marco Passage Ranking 38:44 User aspect of vector search algorithms 43:19 Approximate vs exact nearest neighbor search tradeoffs 47:58 Misconceptions in neural search 52:06 Ranking competitions, idea generation and BERT bi-encoder dream 56:19 Helping wider community through improving search over CORD-19 dataset 58:13 Multimodal search is where vector search shines 1:01:14 Power of building fully-fledged demos 1:04:47 How to combine vector search with sparse search: Reciprocal Rank Fusion 1:10:37 The philosophical WHY question: Jo Kristian’s drive in the search field 1:21:43 Announcement on the coming features from Vespa - Jo Kristian’s Twitter: https://twitter.com/jobergum - Dmitry’s Twitter: https://twitter.com/DmitryKan For the Show Notes check: https://www.youtube.com/watch?v=UxEdoXtA9oM

Update: ZIR.AI has relaunched as Vectara: https://vectara.com/ Topics: 00:00 Intro 00:54 Amin’s background at Google Research and affinity to NLP and vector search field 05:28 Main focus areas of ZIR.AI in neural search 07:26 Does the company offer neural network training to clients? Other support provided with ranking and document format conversions 08:51 Usage of open source vs developing own tech 10:17 The core of ZIR.AI product 14:36 API support, communication protocols and P95/P99 SLAs, dedicated pools of encoders 17:13 Speeding up single node / single customer throughput and challenge of productionizing off the shelf models, like BERT 23:01 Distilling transformer models and why it can be out of reach of smaller companies 25:07 Techniques for data augmentation from Amin’s and Dmitry’s practice (key search team: margin loss) 30:03 Vector search algorithms used in ZIR.AI and the need for boolean logic in company’s client base 33:51 Dynamics of open source in vector search space and cloud players: Google, Amazon, Microsoft 36:03 Implementing a multilingual search with BM25 vs neural search and impact on business 38:56 Is vector search a hype similar to big data few years ago? Prediction for vector search algorithms influence relations databases 43:09 Is there a need to combine BM25 with neural search? Ideas from Amin and features offered in ZIR.AI product 51:31 Increasing the robustness of search — or simply making it to work 55:10 How will Search Engineer profession change with neural search in the game? Get a $100 discount (first month free) for a 50mb plan, using the code VectorPodcast (no lock-in, you can cancel any time): https://zir-ai.com/signup/user

Topics: 00:00 Introduction 01:04 Yury’s background in laser physics, computer vision and startups 05:14 How Yury entered the field of nearest neighbor search and his impression of it 09:03 “Not all Small Worlds are Navigable” 10:10 Gentle introduction into the theory of Small World Navigable Graphs and related concepts 13:55 Further clarification on the input constraints for the NN search algorithm design 15:03 What did not work in NSW algorithm and how did Yury set up to invent new algorithm called HNSW 24:06 Collaboration with Leo Boytsov on integrating HNSW in nmslib 26:01 Differences between HNSW and NSW 27:55 Does algorithm always converge? 31:56 How FAISS’s implementation is different from the original HNSW 33:13 Could Yury predict that his algorithm would be implemented in so many frameworks and vector databases in languages like Go and Rust? 36:51 How our perception of high-dimensional spaces change compared to 3D? 38:30 ANN Benchmarks 41:33 Feeling proud of the invention and publication process during 2,5 years! 48:10 Yury’s effort to maintain HNSW and its GitHub community and the algorithm’s design principles 53:29 Dmitry’s ANN algorithm KANNDI, which uses HNSW as a building block 1:02:16 Java / Python Virtual Machines, profiling and benchmarking. “Your analysis of performance contradicts the profiler” 1:05:36 What are Yury’s hopes and goals for HNSW and role of symbolic filtering in ANN in general 1:13:05 The future of ANN field: search inside a neural network, graph ANN 1:15:14 Multistage ranking with graph based nearest neighbor search 1:18:18 Do we have the “best” ANN algorithm? How ANN algorithms influence each other 1:21:27 Yury’s plans on publishing his ideas 1:23:42 The intriguing question of Why Show notes: - HNSW library: https://github.com/nmslib/hnswlib/ - HNSW paper Malkov, Y. A., & Yashunin, D. A. (2018). Efficient and robust approximate nearest neighbor search using hierarchical navigable small world graphs. TPAMI, 42(4), 824-836. (arxiv:1603.09320) - NSW paper Malkov, Y., Ponomarenko, A., Logvinov, A., & Krylov, V. (2014). Approximate nearest neighbor algorithm based on navigable small world graphs. Information Systems, 45, 61-68. - Yury Lifshits’s paper: https://yury.name/papers/lifshits2009combinatorial.pdf - Sergey Brin’s work in nearest neighbour search: GNAT - Geometric Near-neighbour Access Tree: [CiteSeerX — Near neighbor search in large metric spaces](http://citeseerx.ist.psu.edu/viewdoc/summary?doi=10.1.1.173.8156) - Podcast with Leo Boytsov: https://rare-technologies.com/rrp-4-leo-boytsov-knn-search/ - Million-Scale ANN Benchmarks: http://ann-benchmarks.com/ - Billion Scale ANN Benchmarks: https://github.com/harsha-simhadri/big-ann-benchmarks - FALCONN algorithm: https://github.com/falconn-lib/falconn - Mentioned navigable small world papers: Kleinberg, J. M. (2000). Navigation in a small world. Nature, 406(6798), 845-845.; Boguna, M., Krioukov, D., & Claffy, K. C. (2009). Navigability of complex networks. Nature Physics, 5(1), 74-80.