English-only embedding models for multilingual docs
Text Embeddings: Speaking languages without learning them? tl;dr: some *-en models perform well on other languages too.
Tag: ML
A performant embedding processing pipeline & tutorial for big XML/HTML data dumps
Let's process 1.369.841 single English XML/HTML files and index them with BAAI/bge-base-en-v1.5 embeddings for semantic search! Sounds fun? Let's go!
Semantic Search Tutorial
A full semantic search tutorial about: data mining with requests and beautifulsoup preprocessing in pandas chunking the document text in smaller paragraphs of the right size for the ML model creating embeddings for each chunk calculating the mean embedding for each document saving data as gzipped json (small file size & easy and fast to read in js with pako.js) creating a static web app based on transformers.js on GitHub Pages App here
SemanticFinder - frontend-only live semantic search with transformers.js
Semantic search right in your browser! Calculates the embeddings and cosine similarity client-side without server-side inferencing, using transformers.js and a quantized version of sentence-transformers/all-MiniLM-L6-v2.
Semantic Search with Qdrant, Hugging Face, SentenceTransformers and transformers.js
Create a fully working semantic search stack with only Qdrant as vector database with built-in API and transformers.js using any huggingface model as your frontend-only embedding generator. No additional inference server needed!
Geospatial Vector Search with Qdrant
Using qdrant for querying text data with vector search and geospatial filters without GPU (CPU only)
