📊 Dashboard Preview

⭐ Why OpenLLM Monitor?

🚀 Features

🔔 Enhanced Smart Alerts & Monitoring

• 🚨 Intelligent Alert System - Proactive monitoring with enhanced real-time notifications and interactive bell icon
• 📊 Advanced Performance Alerts - Latency, error rate, and retry monitoring with detailed analysis and visual indicators
• 💰 Smart Cost Alerts - Budget threshold notifications with comprehensive spending insights and trends
• 🔍 Detailed Alert Analysis - In-depth alert analysis with actionable recommendations and impact assessment
• 📈 Enhanced Trend Monitoring - Continuous tracking of key performance metrics with advanced visualizations
• 🎛️ Notification Management - Granular notification controls with test notification capabilities

🧪 Enhanced Model Testing & Experimentation

• 🔬 Redesigned Testing Interface - Completely redesigned model testing with template categories and step-by-step wizard
• 🎯 Template Categories - Organized templates for Quick Start, Development, Creative, and Analysis use cases
• 🔄 Advanced Multi-Model Comparison - Enhanced side-by-side comparison with detailed performance metrics
• 💡 Smart Template Library - Pre-built prompts with categorization, icons, and estimated completion times
• 📊 Real-time Cost Estimation - Accurate cost preview with token counting before running tests
• � Enhanced Batch Testing - Test multiple configurations with progress tracking and results analysis

📱 Modern UI/UX & Enhanced User Experience

• ✨ Beautiful Loading Experience - Animated loading screen with gradient backgrounds and smooth transitions
• 🎨 Enhanced Responsive Design - Fully optimized layouts for desktop, tablet, and mobile devices
• ⌨️ Keyboard Shortcuts - Built-in shortcuts for power users (Cmd/Ctrl+K, R, Escape)
• 🔄 Live Feed Mode - Real-time activity monitoring with toggleable live feed and activity counters
• 🎭 Smooth Animations - CSS animations and transitions for enhanced user experience
• 📊 Interactive Components - Enhanced tables, modals, and interactive elements with improved UX
• 📝 Pre-built prompts for different use cases
• ⚡ Quick Examples - Pre-built prompts for different use cases
• 💰 Cost Estimation - Preview costs before running expensive tests
• 📊 Performance Benchmarking - Compare latency, quality, and costs

📊 Core Monitoring

• 📊 Real-time Request Logging - Monitor all LLM API calls with detailed metrics
• 🔄 Prompt Replay & Comparison - Re-run prompts across different providers/models
• 💰 Cost Tracking & Analysis - Track spending across providers with detailed breakdowns
• ⚡ Performance Monitoring - Latency tracking, retry analysis, and error monitoring
• 🚨 Error Tracking - Comprehensive error analysis and alerting

🎨 Latest UI/UX Enhancements

OpenLLM Monitor has received major UI/UX improvements and feature enhancements!

undefined🚀 What’s New:undefined

• ✨ Beautiful Loading Experience with animated backgrounds
• 🔔 Smart Alerts System with interactive bell notifications
• 📱 Enhanced Mobile Experience with responsive design
• ⌨️ Keyboard Shortcuts for power users
• 🔄 Live Feed Mode with real-time activity monitoring
• 🧪 Redesigned Model Testing with template categories
• 📊 Interactive Components with smooth animations

undefined📖 See Complete Enhancement Guide: Enhanced Features Documentation

🌐 Provider Support

🔧 Integration Options

• 🚀 Zero-code Proxy Servers - Monitor without changing your code
• 📦 SDK Wrappers - Drop-in replacements for popular libraries
• 💻 CLI Monitoring - Track command-line LLM usage
• 🔌 Custom Middleware - Integrate with your existing applications

� Dashboard Features

• 🔴 Real-time WebSocket Updates - Live dashboard with instant updates
• 📊 Comprehensive Analytics - Usage patterns, trends, and insights
• 📤 Export Capabilities - CSV, JSON export for logs and analytics
• 🌍 Multi-Environment Support - Dev, staging, and production environments
• 🎨 Customizable Views - Personalized dashboards and filtering

🚀 Quick Start

🌱 Demo Data Generation (New!)

undefinedWant to showcase your system immediately? Generate comprehensive seed data:

# Windows PowerShell (Recommended)
cd "scripts"
.\generate-seed-data.ps1

# Or use Node.js directly
cd scripts
npm install
node seed-data.js

undefined✨ What you get:undefined

• 🔢 1,000+ realistic LLM requests across 30 days
• 🏢 Multi-provider coverage (OpenAI, Ollama, Mistral, OpenRouter)
• 🎪 7 diverse use cases (coding, analysis, support, creative, etc.)
• 📊 Analytics-ready data for impressive demos
• 💰 Cost tracking with real pricing models
• ⚡ Performance metrics and error patterns

📖 Complete Seed Data Guideundefined | ⚙️ Advanced Configurationundefined

🐳 Docker (Recommended)

Get up and running in less than 2 minutes:

# Clone the repository
git clone https://github.com/prajeesh-chavan/openllm-monitor.git
cd openllm-monitor

# Start with Docker (includes everything)
docker-compose up -d

# Or use our setup script
./docker-setup.sh  # Linux/Mac
.\docker-setup.ps1 # Windows PowerShell

undefined🌐 Access your dashboard: http://localhost:3000

⚡ Manual Setup

git clone https://github.com/prajeesh-chavan/openllm-monitor.git
cd openllm-monitor

# Backend setup
cd backend
npm install
cp ../.env.example .env

# Frontend setup
cd ../frontend
npm install

MONGODB_URI=mongodb://localhost:27017/openllm-monitor
PORT=3001
OPENAI_API_KEY=your-openai-key-here
OLLAMA_BASE_URL=http://localhost:11434

# Terminal 1: MongoDB
mongod

# Terminal 2: Backend
cd backend && npm run dev

# Terminal 3: Frontend
cd frontend && npm run dev

� Installation

🔧 System Requirements

📦 Installation Methods

docker-compose up -d

npm install

docker build -t openllm-monitor .

🚀 One-Click Setup Scripts

undefinedWindows:undefined

# PowerShell (Recommended)
.\docker-setup.ps1

# Command Prompt
docker-setup.bat

undefinedLinux/macOS:undefined

# Make executable and run
chmod +x docker-setup.sh
./docker-setup.sh

undefinedValidation:undefined

# Check if everything is configured correctly
.\docker\docker-validate.ps1  # Windows
./docker/docker-validate.sh   # Linux/Mac

🏗️ Architecture

graph TB
    A[Client Applications] --> B[OpenLLM Monitor Proxy]
    B --> C{LLM Provider}
    C --> D[OpenAI]
    C --> E[Ollama]
    C --> F[OpenRouter]
    C --> G[Mistral AI]

    B --> H[Backend API]
    H --> I[MongoDB]
    H --> J[WebSocket Server]
    J --> K[React Dashboard]

    H --> L[Analytics Engine]
    H --> M[Cost Calculator]
    H --> N[Token Counter]

📁 Project Structure

openllm-monitor/
├── 🎯 backend/                 # Node.js + Express + MongoDB
│   ├── controllers/           # 🎮 API request handlers
│   ├── models/               # 📊 Database schemas & models
│   ├── routes/               # 🛣️ API route definitions
│   ├── middlewares/          # 🔌 Custom middleware (LLM logger)
│   ├── services/             # 🔧 LLM provider integrations
│   ├── utils/                # 🛠️ Helper functions & utilities
│   └── config/               # ⚙️ Configuration management
│
├── 🎨 frontend/               # React + Vite + Tailwind
│   ├── src/components/       # 🧩 Reusable UI components
│   ├── src/pages/            # 📄 Page-level components
│   ├── src/services/         # 🌐 API communication layer
│   ├── src/hooks/            # 🪝 Custom React hooks
│   ├── src/store/            # 🗄️ State management (Zustand)
│   └── public/               # 📂 Static assets
│
├── 🐳 docker/                 # Docker configuration
├── 📚 docs/                   # Documentation & guides
├── 🧪 scripts/               # Setup & utility scripts
└── 📝 README.md              # You are here!

🛠️ Tech Stack

undefinedBackendundefined	undefinedFrontendundefined	undefinedDatabaseundefined	undefinedDevOpsundefined

📊 Usage Examples

🔄 Automatic Monitoring (Zero Code Changes)

# Start the proxy server
npm run proxy

# Your existing code works unchanged!
# All OpenAI calls are automatically logged
const response = await openai.chat.completions.create({
  model: "gpt-3.5-turbo",
  messages: [{ role: "user", content: "Hello!" }]
});

🎯 Direct Integration

// Add to your existing application
const { LLMLogger } = require("openllm-monitor");

const logger = new LLMLogger({
  apiUrl: "http://localhost:3001",
});

// Wrap your LLM calls
const response = await logger.track(async () => {
  return await openai.chat.completions.create({
    model: "gpt-4",
    messages: [{ role: "user", content: "Explain quantum computing" }],
  });
});

📈 Analytics & Insights

// Get comprehensive analytics
const analytics = await fetch("/api/analytics", {
  method: "POST",
  body: JSON.stringify({
    dateRange: "last-7-days",
    providers: ["openai", "ollama"],
    groupBy: "model",
  }),
});

console.log(analytics.data);
// {
//   totalRequests: 1247,
//   totalCost: 23.45,
//   averageLatency: 850,
//   topModels: [...]
// }

🔄 Prompt Replay & Comparison

// Compare the same prompt across providers
const comparison = await fetch("/api/replay/compare", {
  method: "POST",
  body: JSON.stringify({
    prompt: "Write a haiku about coding",
    configurations: [
      { provider: "openai", model: "gpt-3.5-turbo" },
      { provider: "ollama", model: "llama2:7b" },
      { provider: "openrouter", model: "anthropic/claude-2" },
    ],
  }),
});

🎛️ Configuration

⚙️ Environment Variables

Create backend/.env from the template:

cp .env.example .env

undefinedEssential Configuration:undefined

# 🗄️ Database
MONGODB_URI=mongodb://localhost:27017/openllm-monitor

# 🚀 Server
PORT=3001
NODE_ENV=development
FRONTEND_URL=http://localhost:5173

# 🤖 LLM Provider API Keys
OPENAI_API_KEY=sk-your-openai-key-here
OPENROUTER_API_KEY=sk-your-openrouter-key-here
MISTRAL_API_KEY=your-mistral-key-here

# 🦙 Ollama (Local)
OLLAMA_BASE_URL=http://localhost:11434

# 🔐 Security
JWT_SECRET=your-super-secret-jwt-key
RATE_LIMIT_MAX_REQUESTS=100

🎯 Provider Setup Guide

🗄️ Database Setup

🐳 Docker (Recommended)

# Start MongoDB with Docker
docker-compose up -d mongodb

# Access MongoDB Admin UI
open http://localhost:8081  # admin/admin

☁️ MongoDB Atlas (Cloud)

1. Create free account at MongoDB Atlas
2. Create cluster and get connection string
3. Add to .env: MONGODB_URI=mongodb+srv://...

💻 Local Installation

# Download and install MongoDB Community Server
# https://www.mongodb.com/try/download/community

# Or with Chocolatey
choco install mongodb

# Start MongoDB service
net start MongoDB

# Install with Homebrew
brew tap mongodb/brew
brew install mongodb-community

# Start MongoDB
brew services start mongodb/brew/mongodb-community

# Install MongoDB
wget -qO - https://www.mongodb.org/static/pgp/server-6.0.asc | sudo apt-key add -
echo "deb [ arch=amd64,arm64 ] https://repo.mongodb.org/apt/ubuntu focal/mongodb-org/6.0 multiverse" | sudo tee /etc/apt/sources.list.d/mongodb-org-6.0.list
sudo apt-get update && sudo apt-get install -y mongodb-org

# Start MongoDB
sudo systemctl start mongod
sudo systemctl enable mongod

🛠️ Automated Setup Scripts

# Use our setup scripts
./scripts/setup-mongodb.sh     # Linux/Mac
.\scripts\setup-mongodb.ps1    # Windows PowerShell
.\scripts\setup-mongodb.bat    # Windows CMD

🚀 Deployment

🐳 Production Docker

# Production build and deploy
docker-compose -f docker/docker-compose.prod.yml up -d

# With custom environment
docker-compose -f docker/docker-compose.prod.yml --env-file .env.production up -d

☁️ Cloud Deployment

# Build and push to ECR
aws ecr get-login-password --region us-west-2 | docker login --username AWS --password-stdin 123456789012.dkr.ecr.us-west-2.amazonaws.com
docker build -t openllm-monitor .
docker tag openllm-monitor:latest 123456789012.dkr.ecr.us-west-2.amazonaws.com/openllm-monitor:latest
docker push 123456789012.dkr.ecr.us-west-2.amazonaws.com/openllm-monitor:latest

# Deploy with ECS or EKS

# Use DigitalOcean App Platform
doctl apps create --spec .do/app.yaml

# Or deploy to Droplet
docker-compose -f docker/docker-compose.prod.yml up -d

# Frontend only (with separate backend)
cd frontend
npm run build

# Deploy frontend to Vercel
vercel --prod

# Deploy backend separately to Railway/Render

🔧 Environment Configuration

undefinedProduction Environment Variables:undefined

NODE_ENV=production
MONGODB_URI=mongodb+srv://production-cluster/openllm-monitor
JWT_SECRET=super-secure-production-secret
CORS_ORIGIN=https://your-domain.com
RATE_LIMIT_MAX_REQUESTS=1000
LOG_LEVEL=info

undefinedSecurity Checklist:undefined

• Change default passwords
• Use HTTPS in production
• Set up proper CORS
• Configure rate limiting
• Enable MongoDB authentication
• Use environment-specific secrets
• Set up monitoring and alerts

📖 API Documentation

🏥 Health & Status

📊 Logs & Analytics

🔄 Replay & Testing

🌐 Provider Management

� WebSocket Events

// Real-time events
socket.on("new-log", (log) => {
  console.log("New request:", log);
});

socket.on("stats-update", (stats) => {
  console.log("Updated stats:", stats);
});

socket.on("error-alert", (error) => {
  console.log("Error detected:", error);
});

🧪 Testing

🚀 Run Tests

# Backend tests
cd backend
npm test                 # Run all tests
npm run test:watch      # Watch mode
npm run test:coverage   # With coverage

# Frontend tests
cd frontend
npm test                # Run all tests
npm run test:ui         # UI test runner
npm run test:coverage   # With coverage

🎯 Test Categories

• undefinedUnit Tests - Individual component testing
• undefinedIntegration Tests - API endpoint testing
• undefinedE2E Tests - Full user journey testing
• undefinedPerformance Tests - Load and stress testing

📊 Test Coverage

�️ Development

🔧 Development Workflow

# 1. Fork & Clone
git clone https://github.com/prajeesh-chavan/openllm-monitor.git
cd openllm-monitor

# 2. Create Feature Branch
git checkout -b feature/amazing-feature

# 3. Start Development
npm run dev:all  # Start all services

# 4. Make Changes & Test
npm test
npm run lint

# 5. Commit & Push
git commit -m "feat: add amazing feature"
git push origin feature/amazing-feature

# 6. Create Pull Request

🎯 Project Status

🏗️ Contributing Areas

We welcome contributions in these areas:

• 🐛 Bug Fixes - Help us squash bugs
• ✨ New Features - Implement requested features
• 📚 Documentation - Improve guides and docs
• 🧪 Testing - Add more test coverage
• 🎨 UI/UX - Enhance user experience
• 🚀 Performance - Optimize speed and efficiency

🤝 Contributing

We love contributions! Here’s how you can help make OpenLLM Monitor even better:

🚀 Quick Contribution Guide

1. undefined🍴 Fork the repository
2. undefined🌟 Star the project (if you like it!)
3. undefined🔧 Create your feature branch (git checkout -b feature/AmazingFeature)
4. undefined💾 Commit your changes (git commit -m 'feat: Add some AmazingFeature')
5. undefined� Push to the branch (git push origin feature/AmazingFeature)
6. undefined🎯 Open a Pull Request

🎯 Ways to Contribute

🏷️ Commit Convention

We use Conventional Commits:

feat: add new dashboard widget
fix: resolve login issue
docs: update API documentation
test: add unit tests for analytics
refactor: optimize database queries
chore: update dependencies

🎖️ Contributors

Thanks to all the amazing people who have contributed to this project!

🆘 Support & Community

� Get Help

🛠️ Troubleshooting

undefinedCommon Issues:undefined

# Check if MongoDB is running
docker ps | grep mongo

# Restart MongoDB
docker-compose restart mongodb

# Check logs
docker-compose logs mongodb

# Find what's using the port
netstat -tulpn | grep :3001

# Kill the process
kill -9 <PID>

# Or change port in .env
PORT=3002

# Check Ollama status
ollama ps

# Restart Ollama
ollama serve

# Check logs
tail -f ~/.ollama/logs/server.log

📚 Resources

• 📖 User Guide - Complete usage guide with enhanced UI features
• 🎨 Enhanced Features Guide - Latest UI/UX improvements and new features
• 🚀 Quick Start - Get running in 5 minutes
• 📊 Features Overview - Comprehensive feature documentation
• 🔧 API Docs - Full API reference
• � Development Guide - Development status and technical details
• �🐳 Docker Guide - Docker setup and deployment
• 🧪 Test Models Guide - Enhanced model testing interface
• � Smart Alerts - Intelligent monitoring and notifications
• �🛠️ Troubleshooting - Common issues and solutions
• 📝 Changelog - All recent improvements and changes

� License

This project is licensed under the MIT License - see the LICENSE file for details.

undefinedCopyright © 2024-2025 Prajeesh Chavanundefined

MIT License

Copyright (c) 2024-2025 Prajeesh Chavan

Permission is hereby granted, free of charge, to any person obtaining a copy
of this software and associated documentation files (the "Software"), to deal
in the Software without restriction, including without limitation the rights
to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
copies of the Software, and to permit persons to whom the Software is
furnished to do so, subject to do so, subject to the following conditions:

The above copyright notice and this permission notice shall be included in all
copies or substantial portions of the Software.

THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT.

👨‍💻 About the Creator

undefinedAbout Me:undefined
I’m a passionate full-stack developer with expertise in modern web technologies and AI/ML systems. I built OpenLLM Monitor to solve the real-world challenge of monitoring and optimizing LLM usage across different providers. This project represents my commitment to creating tools that help developers work more efficiently with AI technologies.

undefinedSkills & Technologies:undefined

• 🚀 Full-Stack Development (React, Node.js, MongoDB)
• 🤖 AI/ML Integration & LLM Applications
• 🐳 DevOps & Cloud Deployment (Docker, AWS)
• 📊 Data Analytics & Visualization
• 🔧 System Architecture & API Design

undefinedConnect with me if you have questions about the project, want to collaborate, or discuss opportunities!

🚀 Project Journey

This project represents months of dedicated development and continuous improvement. Here’s what makes it special:

• undefined🔬 Research-Driven: Extensive research into LLM monitoring needs and best practices
• undefined🏗️ Built from Scratch: Every component carefully designed and implemented
• undefined🎯 Problem-Solving: Addresses real-world challenges faced by LLM developers
• undefined📈 Continuous Evolution: Regular updates and feature enhancements
• undefined🌍 Community-First: Open source with detailed documentation and support

undefinedIf you find this project valuable, please:undefined

• ⭐ Star the repository to show your support
• 🤝 Connect with me on LinkedIn or Twitter
• 💬 Share your feedback or suggestions
• 🎯 Consider hiring me for your next project!

🌟 Star History

🎯 What’s Next?

🎖️ Project Roadmap

• ✅ v1.0 - Core monitoring and analytics
• 🚧 v1.1 - Advanced filtering and exports
• 📋 v1.2 - Mobile app and notifications
• 🔮 v2.0 - AI-powered insights and recommendations