Large language models (LLMs) have emerged to become an integral part of the software stack. They are available through APIs from providers like Cohere, Google Cloud, and OpenAI or as open-source models hosted on Hugging Face.
However, utilizing LLMs isn’t merely about sending prompts. Developers must consider aspects like parameter adjustments, prompt augmentation, and response moderation. LLMs are stateless, so developers must maintain conversational history for context, possibly using databases for long-term storage.
Furthermore, there isn’t a universal LLM solution. Applications might require various specialized models, complicating integration and increasing development complexity.
LangChain is becoming the tool of choice for developers building production-grade applications powered by LLMs. It has a diverse and vibrant ecosystem that brings various providers under one roof, including Google’s PaLM2 large language model.
In this tutorial, we will walk through the steps of building a LangChain application backed by the Google PaLM 2 model.
Setting up the environment
Visit Google MakerSuite and create an API key for PaLM.
IDG
In the terminal, create a Python virtual environment and activate it.
python -m venv venv source venv/bin/activate
Create an environment variable to store the PaLM API key.
export GOOGLE_API_KEY=YOUR_API_KEY
Install the following Python modules.
pip install google-generativeai pip install langchain pip install pypdf pip install jupyter
Accessing the PaLM API
Start a new Jupyter Notebook (on your local workstation or in Google Colab) and run the following code.
import google.generativeai as palm import os google_api_key=os.getenv(‘GOOGLE_API_KEY’) palm.configure(api_key=google_api_key) prompt = ‘Explain the difference between effective and affective with examples’ completion = palm.generate_text( model=’models/text-bison-001′, prompt=prompt, temperature=0.1 ) print(completion.result)
The program starts by importing the Python modules and then gets the API key from the environment variable. It invokes the generate_text method by setting the model to models/text-bison-001 and passing the prompt variable.
The temperature variable defines the predictability of the model. The values closer to zero will make the output deterministic and predictable.
It generates the following output.
IDG
Because the prompt instructed the model to explain the difference with an example, the model responded with a detailed response.
Now, let’s repeat the same exercise with LangChain.
Start a new Jupyter Notebook and run the code below.
from langchain.embeddings import GooglePalmEmbeddings from langchain.llms import GooglePalm import google.generativeai import os google_api_key=os.getenv(‘GOOGLE_API_KEY’) llm = GooglePalm(google_api_key=google_api_key) llm.temperature = 0.1 prompts = [‘Explain the difference between effective and affective with examples’] llm_result = llm._generate(prompts) print(llm_result.generations[0][0].text)
The code is not only cleaner but simple to understand. It initializes the llm variable by pointing to the Google PaLM model. It then sets the temperature variable.
In LangChain, the prompts parameter is a Python list. You can send multiple prompts at once and get back multiple generations.
Since we only passed one prompt, we accessed the text property of the first generation.
IDG
As we can see, the output is the same as the previous program.
The objective of this tutorial was to introduce PaLM API and the seamless integration with LangChain. The advantage of this approach is the ability to swap the LLM with minimal changes to the code. With LangChain, the LLM becomes one of the “links” of the chain, which can be easily replaced.
In the next part of this series, we will build a LangChain Q&A program based on a custom PDF document. Stay tuned.
Copyright © 2023 IDG Communications, Inc.
