Photo: Kim Stiver/Pexels

More LLM/AI Tooling

Run, serve and experiment with LLMs and other AIs

Much like the 90’s it is a wonderful time to be a software developer, tech person or coder, just that instead of the web, the current awesome thing (and it is awesome ) is LLM/AIs, along with them though there are tons of new things one needs to learn to do things like host your own LLM, make demos for your new startup idea or just tinker around and make a sentient AI ( a pet project of mine ), so while much of what I’ll talk about will age poorly, think of this as a snapshot mid 2024 of some choice AI tooling.

I recently wrote about HuggingFace and Langchain,not required as they 
will also be covered here, but they go more indepth if you want to check 
them before/after.

Get yourself a cutting edge LLM AI and run it in 2 minutes flat !

The easiest/fastest/no stress way to run an AI, and also a great introduction to tooling is via Google’s Colab (hosted notebooks), Unsloth (LLM acceleration/tooling ), using Meta’s llama 3.1-8B ( The AI/LLM ), how easy ?

# See here for GNU licence 
# https://colab.research.google.com/drive/1T-YBVfnphoVc8E2E854qF3jdia2Ll2W2
# But basically 3 lines 

!git clone https://github.com/unslothai/studio > /dev/null 2>&1
with…