Lecture 10 - Building Streamlit Apps with LLMs | Multimodal AI with Image Input & Base64 Encoding

Learn to build production-ready Streamlit applications powered by LLMs! This lecture covers creating interactive AI apps with both text and image inputs, understanding multimodal models, and mastering Base64 encoding for multimedia processing.
📚 What You'll Learn:
Building LLM-Powered Streamlit Apps

Creating a Counselor AI application from scratch
Setting up page configuration and custom titles
Implementing sidebars for better UX
User input handling with text areas and buttons
Displaying LLM responses with proper formatting
Managing system roles and user queries

Understanding Multimodal Models

What makes a model "multimodal"?
Text vs Image vs Audio vs Video inputs
Which LLM models support multiple formats
GPT model capabilities across different media types

Image Token Calculation & Pricing

How tokens are calculated for images
Understanding pixels and image dimensions (360p, 720p, 1080p, 4K)
What is a pixel? The smallest unit explained
Image quality vs file size vs token cost
OpenAI pricing model for image inputs
Cost optimization strategies (resize to 720x720)
Low vs High quality image processing

Working with Images in LLMs

Why images need Base64 encoding
Converting local images to Base64 format
Proper message formatting for image inputs
Combining text and image queries
API requirements for multimedia data
Local models vs API-based models (Ollama vs OpenAI)

Practical Implementation

Creating a text-based Q&A app with custom system roles
Building templates with ChatPromptTemplate
Chain creation and invocation
Error handling and debugging
Testing in Jupyter notebooks before Streamlit deployment

💡 Key Concepts:
Multimodal Definition:
A model that supports more than one input format:

Text only = Not multimodal
Text + Image = Multimodal
Text + Image + Audio + Video + PDF = Multimodal

Token Calculation for Images:

Text: ~4 characters = 1 token
Images: Based on 32x32 pixel patches
Quality levels affect token count (low/high)
Maximum patch limit: 1536 (auto-scales if exceeded)

Base64 Encoding:
Essential for sending images to LLM APIs. Converts image files into string format that APIs can understand. Not needed for local models like Ollama.
🛠️ Tools & Technologies:

Streamlit for UI
LangChain for LLM integration
OpenAI GPT models
Base64 library (built-in Python)
ChatPromptTemplate for structured prompts

📂 Project Structure:
llm_app/
├── app1.py (Counselor AI)
├── template.ipynb (Testing ground)
└── multimedia_template.ipynb (Image handling)
🎯 Best Practices:

Always test logic in notebooks first
Use UV instead of pip for faster package management
Resize images to 720x720 for cost optimization
Start with low-quality images, upgrade if needed
Enable "run on save" in Streamlit settings
Check documentation rather than memorizing syntax

⚠️ Important Notes:

Base64 encoding required for API-based LLMs only
Internet images are already Base64 encoded
Local images must be converted before sending
Token costs vary by image size and quality
Always validate image format before processing

📖 Homework Assignment:
Practice with different media types:

Images (covered in this lecture)
Audio files
PDF documents
Video inputs

Experiment with LangChain documentation examples for each format.
⏰ Duration: ~78 minutes
🔗 Resources:

LangChain multimodal documentation
OpenAI pricing calculator
Base64 encoding reference
Streamlit component library

Perfect for developers building AI applications with visual understanding, chatbots that process images, document analysis tools, and multimodal AI assistants.
#Streamlit #LLM #MultimodalAI #Python #OpenAI #LangChain #Base64 #ImageProcessing #AIApp #ChatGPT #GenAI #MachineLearning #WebDevelopment #TokenOptimization