GEN-AI SOFTWARE STACK

software stack

A software stack, as we know, is a collection of tools used to build a software application. Similarly, developing a GENAI application also requires a specific software stack. To know more about software stacks, you can follow this link.

To better understand software stacks, you can follow the provided link. To develop our GENAI application, we should first know which software stacks we need to use.

MODELS AND API

We have different categories of GENAI models broadly it can be defined under four categories.

• Closed LLMs
  • PaLM 2 by google
  • LLaMA by Meta
• Open LLMs
  • HuggingChat
  • Dolly by Databricks
  • StableLM by stability AI
  • OpenLLaMA by UC Berkley
• Image Models
  • Midjourney
  • DALLE-3 2 by OpenAI
  • StableDiffusion by stability.ai
  • Runway
• Music Models
  • MusicLM by google
  • MusicGen by Meta

Vector Dtabase

A vector database is a specialized type of database designed to store and manage high-dimensional vector data. Unlike traditional databases that store scalar values (like numbers or text), vector databases handle complex data points represented as vectors1. These vectors can be thought of as arrows in a multi-dimensional space, capturing various characteristics or qualities of the data.

Vector databases are particularly useful for applications involving similarity searches. For example, they can be used to find images similar to a given image, recommend products based on user preferences, or even search for text that matches a particular context2.

How Vector Databases Work

Vector databases use Approximate Nearest Neighbor (ANN) search algorithms to quickly find the closest matching vectors. This is crucial for handling large datasets efficiently2. By clustering vectors based on similarity, these databases can perform low-latency queries, making them ideal for AI applications.

The Synergy of Vector Databases and Gen-AI

Combining vector databases with generative AI can unlock powerful capabilities:

Efficient Data Handling: Generative AI models often require large amounts of data for training. Vector databases can manage and organize this data effectively, enabling faster and more efficient model training1.

Enhanced Search and Retrieval: Vector databases can efficiently retrieve relevant data points for generative AI models, improving the accuracy and relevance of generated content1.

Personalization: By leveraging vector databases, generative AI can provide highly personalized recommendations and content, tailored to individual user preferences. Some examples of vector databases are given below :

1. Pinecone
2. Chroma
3. Qdrant

LLM Frameworks

LLM frameworks allows developer to build complex software using LLMs that can execute a sequence of tasks and access external APIs to perform more complex tasks. Frameworks LLMs to interact with other data sources and APIs and allow them to interact with the software environment. In some of the situation LLMs require to interact with external APIS such as WolframAlpha specifically to solve mathematical problems. Some examples of LLM frameworks are

• Langchain
• LlamaIndex
• Anarchy

ai Deployment

Deployment infrastructure is a combination of services provided by vendors, which allow GENAI applications to be deployed on them. Microsoft provides GENAI deployment services under Azure OpenAI Services. Other examples of deployment infrastructure are Vertex AI by Google and Hugging Face Inference Endpoints.