Source : Pexels/Pixabay

LangChain

What | is | it ?

One of the characteristics of new technologies is that sometimes the new products and services are not immediately easy to grasp, you are telling me you can sell a book online !? Why, how, where ? I think many people asked these questions at the dawn of the internet, but more relevant the tools behind the scene need some learning too.

LangChain sits somewhere between a new product type: LLMs/AIs as software components and the new set of tooling required to integrate them into your code or project, so let’s check it out.

Bring your own AI

Before we start you need API Keys for LLMs or other AIs you plan to use, that is LangChain doesn’t include the AI proper but facilitates using them, I opted for the tried and tested OpenAI API, you do need to have some credits ( $5.00 should work ).

Basic Chains

So what’s a chain anyways ? In Langchains parlance it is a sequence of operations that involve processing inputs, interacting with a language model like an LLM or other AIs, and handling the outputs.

The metaphor of a chain seems a bit forced at this point, but imagine that each process is a link in a chain.

And a simple code example of a basic chain :

from langchain_core.messages import HumanMessage, SystemMessage
from langchain_core.output_parsers import StrOutputParser

# INPUT PROCESSING :
query = "t-shirt" 

# LLM CALL :
messages = [
    SystemMessage(content="Translate the following from English 
     into Spanish"),
    HumanMessage(content=query),
]


# OUTPUT PROCESSING :
parser = StrOutputParser()
result = model.invoke(messages)
print(parser.invoke(result))

# >>> camiseta

Couple of things :

There’s considerable set up to get this bit of code to run, consult your chosen LLM/AI API reference, you also need to authenticate yourself with LangChain.

Speaking of hardships I found the docs more confusing than helpful at times, changes and complexity compound the issue and I kept on finding ways to do things (important things) that were never fully explained or delegated to a code sample somewhere in an API reference page, here for instance is a minimal chat prompt ( which I literally stumbled upon while researching memory implementations )…

from langchain_openai import OpenAI, ChatOpenAI
from langchain.chains import ConversationChain

model = ChatOpenAI(model="gpt-4")
conversation = ConversationChain(llm=model, verbose=True)
response = conversation.invoke({"input": "Whats the most common plant in 
the world ?"})

print(response)

Which gives you:

Prompt after formatting:
The following is a friendly conversation between a human and an AI. 
The AI is talkative and provides lots of specific details from its context. 
If the AI does not know the answer to a question, 
it truthfully says it does not know.

Current conversation:

Human: Whats the most common plant in the world ?
AI:

> Finished chain.
{'input': 'Whats the most common plant in the world ?', 
'history': '', 
'response': "The most common plant in the world is arguably 
the flowering plant species known as grass..... 

You might also be thinking what’s the point ? You could do this via the native API, with OpenAIs API for instance you’d simply do something like this:

from openai import OpenAI
client = OpenAI()

completion = client.chat.completions.create(
  model="gpt-4",
  messages=[
    {"role": "user", "content": "Translate the following 
      from English into Spanish: t-shirt"}
  ]
)

print(completion.choices[0].message.content)

# >>> camiseta

The next sections deal with more complex cases that might sell you on the technological debt, but even at this early examination LangChain provides a standardized framework for interacting with multiple LLMs/AIs which in itself is valuable.

Bigger Chains

In a way the biggest conceptual change can be exemplified with a longer chain, for instance you give the LLM some text in spanish, you translate that into english and then perform some sort of sentiment analysis, or in general form :

Note that there could be extensive processing before and after interacting with each AI, which could also be a different type of AI.

And a code sample:

from langchain_core.output_parsers import StrOutputParser
from langchain_core.prompts import ChatPromptTemplate

translate_prompt = ChatPromptTemplate.from_template
("Translate the following text from Spanish to English: {spanish}")
analysis_prompt = ChatPromptTemplate.from_template
("Analyze the sentiment of the following English text: {input_text}")

chain = translate_prompt | model | analysis_prompt | model | StrOutputParser()

chain.invoke({"spanish": "Hoy me fue muy bien !"})

# >>>The sentiment of the text "Today went really well for me!" is positive.

Note I am using somehow more advanced syntax like templates which structure prompts and pipes which chain runnables/tasks and not doing any processing in between links to keep things simple.

Tools, structured data and Agents

Graduating AIs to software components is next, tools for instance allow you to connect external information, a special link in the chain if you will, for instance getting financial news articles about a company and doing basic sentiment analysis on them:

Note that a tool or group of tools can be placed anywhere in the chain, here it just happens to be the first step.

from langchain_community.tools.yahoo_finance_news
import YahooFinanceNewsTool
from langchain_core.output_parsers import StrOutputParser
from langchain_core.prompts import ChatPromptTemplate

tool = YahooFinanceNewsTool()
articles = tool.run("NVDA")
print(articles,'\n\n')

analysis_prompt = ChatPromptTemplate.from_template
("Analyze the sentiment of the following articles: {input_text}")

chain = analysis_prompt | model | StrOutputParser()
chain.invoke({"input_text": articles})


#>>> Articles:
#Top S&P 500 Stocks of 2024: SMCI, CEG, NVDA Ride AI Wave
#It's no secret that AI bullishness has pushed many stocks higher in 2024


#Analyst Says Advanced Micro Devices, Inc. 
#(AMD) is Expensive Compared With NVDA,


#Sentiment:
#The sentiment of the first article is positive
# The sentiment of the second article is negative 
# towards Advanced Micro Devices 

#Check the docs for more tools:
# https://python.langchain.com/v0.2/docs/integrations/tools/

Structured Data

Another consideration when interacting programatically with LLMs/AIs is the structuring or parsing of results (see the above sample for input variables ), which LangChain also supports, let’s say we want some random floats…

from langchain.output_parsers import CommaSeparatedListOutputParser

output_parser = CommaSeparatedListOutputParser()

getFloat = model.invoke([("human", "Generate a random f
loat number between 0 and 1")])
print(float(getFloat.content))


getFloats = model.invoke([("human", "Generate two random 
float number between 0 and 1")])

result = output_parser.parse(getFloats.content)
for item in result:
    print(float(item))

# >>>>
# 0.7568392831
# ...
# 0.7632895
# 0.3547698

The first prompt just needs some type conversion, while the second one needs a bit of extra parsing to get the two float values, this is just the start though, you can get a variety of structured and parsed outputs.

Agents

Moving up on the complexity and functionality ladder there are agents which ( from the docs ): are systems that use LLMs as reasoning engines to determine which actions to take and the inputs to pass them, in other words you can delegate some logic from your program to the AI, usually as a prompt !

I’ve abandoned the chain metaphor as the system used to create agents is a graph and there’s a specific library called LangGraph, (but do note that the complexity escalates considerably), so we’ll make a simple (ish) agent without explicitly invoking a graph, the task here for the agent is to turn on the AC depending on the current temperature at a certain location which it doesn’t have.

from langchain_core.prompts import ChatPromptTemplate, MessagesPlaceholder
from langchain_openai import OpenAI, ChatOpenAI
from langchain.agents import load_tools, AgentExecutor, 
create_tool_calling_agent
from langchain_core.output_parsers import StrOutputParser

model = ChatOpenAI(model="gpt-4")

tools = load_tools(["openweathermap-api"], model)

prompt = ChatPromptTemplate.from_messages(
    [
        ("system", "You are a very powerful assistant, 
          but don't know current weather, give the temperature in 
          °C once you know it"),
        ("user", "What is the current temperature in {location}?"),
        MessagesPlaceholder(variable_name="agent_scratchpad"),
    ]
)

agent = create_tool_calling_agent(model, tools, prompt)
agent_executor = AgentExecutor(agent=agent, tools=tools, verbose=True)

# Get the temperature for a specific location
response = agent_executor.invoke({"location": "Mexico City"})


# Extract and print the temperature
weather_data = response["output"]
print(weather_data)


reasoning_prompt = ChatPromptTemplate.from_template("Turn the AC on 
if the tempereature is above 25°C else leave it off: {weather_data}")

chain = reasoning_prompt | model | StrOutputParser()
chain.invoke({"weather_data": weather_data})

And here’s the output:

> Entering new AgentExecutor chain...

Invoking: `open_weather_map` with `Mexico City`

In Mexico City, the current weather is as follows:
Detailed status: scattered clouds
Wind speed: 7.2 m/s, direction: 30°
Humidity: 12%
Temperature: 
  - Current: 27.75°C
  - High: 27.96°C
  - Low: 27.75°C
  - Feels like: 26.43°C
Rain: {}
Heat index: None
Cloud cover: 40%The current temperature in Mexico City is 27.75°C.

> Finished chain.
The current temperature in Mexico City is 27.75°C.
Turn the AC on.

So we ask the LLM to use a tool ( getting the weather ) and when we get the response we then ask again the LLM to reason and make a decision based on the tools response data, you could even go meta and change the reasoning logic in another prompt without writing a single line of code, pretty cool no ?

Agents in LangGraph are not as simple due to the myriad other options (like memory, response type) and built in agents like the ReAct (Reasons and Acts) agent we just cobbled, its a bit of a rabbit hole, check the docs for more:

LangGraph How-Tos: https://langchain-ai.github.io/langgraph/how-tos/

Memory

One last feature you might want to check is memory, which like agents can get deep and complex depending on what you want, message memory would look like this :

So you can have a chat session with an ID, and store both the prompts and responses and use them at any point to continue the conversation or chain, they do get complex, but here’s a simple example of storing the chat memory :

from langchain_openai import OpenAI, ChatOpenAI
from langchain.chains import ConversationChain
from langchain.memory import ConversationBufferMemory

model = ChatOpenAI(model="gpt-4")

# Initialize the memory
memory = ConversationBufferMemory()

conversation = ConversationChain(llm=model, verbose=True, memory=memory)
response1 = conversation.invoke({"input": "Whats 2 + 2 ?"})
print(response1)

response2 = conversation.invoke({"input": "Whats 4 + 4 ?"})
print(response2)

print("\nConversation History:")
print(memory.buffer)

> Entering new ConversationChain chain...
Prompt after formatting:
The following is a friendly conversation between a human and an AI. 
The AI is talkative and provides lots of specific details from its context.
If the AI does not know the answer to a question, 
it truthfully says it does not know.

Current conversation:

Human: Whats 2 + 2 ?
AI:

> Finished chain.
{'input': 'Whats 2 + 2 ?', 'history': '', 
'response': 'The sum of 2 + 2 is 4.'}


> Entering new ConversationChain chain...
Prompt after formatting:
The following is a friendly conversation between.... 

Current conversation:
Human: Whats 2 + 2 ?
AI: The sum of 2 + 2 is 4.
Human: Whats 4 + 4 ?
AI:

> Finished chain.
{'input': 'Whats 4 + 4 ?', 
'history': 'Human: Whats 2 + 2 ?\nAI: The sum of 2 + 2 is 4.', 
'response': 'The sum of 4 + 4 is 8.'}

Conversation History:
Human: Whats 2 + 2 ?
AI: The sum of 2 + 2 is 4.
Human: Whats 4 + 4 ?
AI: The sum of 4 + 4 is 8.

For more complex memory Message_History: 

https://python.langchain.com/v0.2/docs/how_to/message_history/

Miscellaneous

There are a host of additional tooling and features scattered across langChains libraries and APIs and I’ll just mention some that stand out due to their practicality or potential , but I suggest you read the Conceptual Guide , how-to’s, and tutorials for the full picture…

Streaming Support All the examples I’ve made here are simple requests (prompt/response), but LLMs and other AIs can return information over a continuous period which gets complex as you have more LLMs interacting, so you can stream and async stream responses, there’s also ample callback support.

LangServe Let’s you deploy your langChain applications as REST APIs

Multimodal inputs If you want to combine text and images as part of your prompts/inputs.

RAG (Retrieval Augmented Generation) support via document loaders, text-splitters, embeddings/vector stores, retrievers along with the rest of the LLM/AI chain we’ve been discussing.

RAG in a nutshell allows you to add your own data or external data in a 
sophisticated way to your AI/LLM so it can be incorporated as context. 

Alternatives

The main alternative here is using native APIs, if you look at the API reference from OpenAI for instance , you will likely see an overlap in functions and maybe an easier way to accomplish what you want.

Having said that you could also go custom, it was my experience ( yours might be different ) that the technological debt and steep learning curve required from incorporating LangChain was not worth it vs just cobbling my own prompt template for instance (hello fstrings ?).

Yet as some of us early web/app/crypto developers can remember the early tooling companies and libraries usually are the ones that prevail, (React/Angular in web/app development and Open Zepellin in contracts for instance) and LangChain has a sort of niche if you want a multi provider general LLM/AI library.

HuggingFace which I recently wrote about has some overlap in functions but
has also a completely different focus (hosting/serving models), but could
easily branch out/sideways into more specific tooling, Nvidia could buy and
consolidate both or some other startup could take the lead in AI tooling,
this goes to say that it is still early and AIs/LLMs are a moving target.

Takeaways

You might be tired of hearing that these are early days for AI, and maybe that’s not entirely true anymore as there are Billion dolar companies out there and a lots of products, startups and research ( maybe it’s just a an early wave ), but as for tooling, that is the things needed to make AI products it is well still early days.

As such tooling is evolving as we speak and the cost for the developer is to constantly be checking out all the new stuff and learning how to use it, all while the tooling itself is being actively developed, so yeah good times.

I had mostly a great time checking out LangChain thinking about the possibilities and making small working examples but can’t neglect to mention that it was very rough at times.

Practically speaking the tagline “Applications that can reason” might not be there yet, or rather it’s probably there or will be there if you commit to learning the tooling which needs some other considerations like alternatives and ROI, but it is as good a time to jump in as any and LangChain is a good place to check out cutting edge LLM app development.

Thanks for reading.