Embeddings

Vector dimension precision effect on cosine similarity

Experiments, AI, Embeddings / By

Experiment Outline Question: How does reducing the precision of vector components to various extents (half, third, quarter, fifth) using different methods (toFixed, Math.round) affect the cosine similarity between two vectors? Hypothesis: Reducing the precision of vectors will alter the cosine similarity, with more significant reductions leading to larger differences. The method of precision reduction might …

Vector dimension precision effect on cosine similarity Read More »

Enhancing Large Document Summarization with Embeddings and Semantic Similarity

Leave a Comment / Embeddings, AI, AI Your Business, GPT-3, GPT-4, LLM / By

I’m excited to share a powerful method for accurately summarizing large documents that exceed the input size limit of GPT-4. I’ve designed a diagram to visually illustrate this approach, which leverages embeddings to check the semantic similarity of summarized blocks (chunks). This iterative process ensures that each block reaches a desired similarity threshold before being …

Enhancing Large Document Summarization with Embeddings and Semantic Similarity Read More »

What are Embeddings: A Simple Guide to Understanding Them

Leave a Comment / Embeddings, AI, Education / By

Have you ever wondered how computers can understand complex things like human language or images? The secret is something called “embeddings.” But what are embeddings, and why are they important? In this blog post, we’ll explain embeddings in a simple, easy-to-understand way. What are Embeddings? Imagine you’re trying to explain the concept of an apple …

What are Embeddings: A Simple Guide to Understanding Them Read More »

Introducing Smart Chat: Transform Your Obsidian Notes into Interactive AI-Powered Conversations with GPT-4 & Embeddings

Leave a Comment / Obsidian, AI, Embeddings, GPT-3, Personal AI, PKM, Productivity / By

Update: Now with GPT-4 and GPT-3.5 support. Hey there, fellow Obsidian users! I’m excited to introduce you to Smart Chat, an amazing new feature of Smart Connections that will revolutionize how you interact with the notes in your Obsidian vault. If you’re unfamiliar with Obsidian, it’s an incredible note-taking platform that helps you store, manage, and connect your …

Introducing Smart Chat: Transform Your Obsidian Notes into Interactive AI-Powered Conversations with GPT-4 & Embeddings Read More »

Q&A with ChatGPT + Embeddings – Basic & HyDE Examples

Leave a Comment / Embeddings, AI, AI Your Business, ChatGPT, Conversational AI, GPT-3, LLM / By

One of the most sought-after AI systems is one where you can easily query a knowledgebase via a ChatGPT-like interface. Basic Q&A with Embeddings Since you can’t fine-tune the newer, more conversational models like the ChatGPT API, embeddings are how you “train” them on your proprietary knowledge base. To create your embedding system, you must …

Q&A with ChatGPT + Embeddings – Basic & HyDE Examples Read More »

Alternatives to Fine-tuning

Leave a Comment / AI, ChatGPT, Embeddings, GPT-3, LLM / By

People commonly refer to fine-tuning when attempting to address their problems with AI. In a recent conversation with an OpenAI Ambassador, I was told that they “convince 90% not to use fine-tuning to solve their problem,” confirming my hunch that fine-tuning is one of the most misunderstood tools in AI. Fine-tuning is often more complicated and less …

Alternatives to Fine-tuning Read More »

Revolutionizing Search: How Hypothetical Document Embeddings (HyDE) Can Save Time and Increase Productivity

Leave a Comment / AI, ChatGPT, Embeddings, GPT-3, LLM / By

Tools like the OpenAI Embeddings API are changing the way search is performed. Instead of matching keywords, a language model converts text into vectors. These vectors represent the text using hundreds or even thousands of dimensions mathematically. This way, when searching, matches can be made based on the underlying intent of the query instead of just keywords. …

Revolutionizing Search: How Hypothetical Document Embeddings (HyDE) Can Save Time and Increase Productivity Read More »