A sophisticated voice analysis application that detects emotions from speech by analyzing both what you say (transcription) and how you say it (vocal tone). Available in two versions: a clean minimal implementation and a luxury dark-mode experience.
- Overview
- Version Comparison
- Visual Architecture
- Dependencies Explained
- Installation
- Usage
- How It Works
- Project Structure
- Contributing
- License
This project demonstrates the power of combining speech-to-text AI, audio signal processing, and natural language understanding to create an empathetic AI companion that truly "hears" you - not just your words, but your emotional state.
- ποΈ Voice Recording - Capture audio through your microphone
- π Speech Transcription - Convert speech to text using OpenAI's Whisper
- π΅ Tone Analysis - Extract vocal features using librosa
- π§ Sentiment Analysis - Detect emotions from both text and voice
- π AI Responses - Generate empathetic recommendations
- π Session History - Track emotional trends over time
A clean, educational implementation perfect for learning.

# Focus: Simplicity and clarity
features = [
"Basic sentiment analysis",
"Simple recording interface",
"Text-only emotion detection",
"Minimal UI with Streamlit defaults"
]
best_for: "Beginners, learning the basics, quick prototypes"
complexity: "β"
A production-ready application with professional design.
# Focus: User experience and depth
features = [
"Advanced tone analysis (MFCC, spectral centroid, tempo)",
"Dark luxury aesthetic with custom CSS",
"Session history with trend visualization",
"Text vs. tone mismatch detection",
"JSON export functionality",
"8+ vocal feature extraction",
"Animated UI components"
]
best_for: "Portfolio projects, production deployment, user research"
complexity: "ββββ"βββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββ
β USER INTERFACE LAYER β
β βββββββββββββββ βββββββββββββββ βββββββββββββββ βββββββββββββββ β
β β Streamlit β β Custom β β Recording β β Results β β
β β App β β CSS β β Controls β β Display β β
β βββββββββββββββ βββββββββββββββ βββββββββββββββ βββββββββββββββ β
βββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββ
β
βΌ
βββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββ
β AUDIO PROCESSING LAYER β
β βββββββββββββββ βββββββββββββββ βββββββββββββββ βββββββββββββββ β
β β SoundDevice β β NumPy β β SciPy β β TempFile β β
β β Recording β β Arrays β β WAV I/O β β Storage β β
β βββββββββββββββ βββββββββββββββ βββββββββββββββ βββββββββββββββ β
βββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββ
β
βΌ
βββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββ
β FEATURE EXTRACTION LAYER β
β βββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββ β
β β librosa β β
β β βββββββββββββββ βββββββββββββββ βββββββββββββββ β β
β β β Energy β β Pitch β β MFCC β β β
β β β Analysis β β Variabilityβ β Features β β β
β β βββββββββββββββ βββββββββββββββ βββββββββββββββ β β
β β βββββββββββββββ βββββββββββββββ βββββββββββββββ β β
β β β Tempo/ β β Spectral β β Zero β β β
β β β BPM β β Centroid β β Crossings β β β
β β βββββββββββββββ βββββββββββββββ βββββββββββββββ β β
β βββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββ β
βββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββ
β
βΌ
βββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββ
β AI SERVICES LAYER β
β βββββββββββββββββββββββββββ βββββββββββββββββββββββββββ β
β β OpenAI Whisper β β OpenAI GPT-3.5 β β
β β βββββββββββββββββββ β β βββββββββββββββββββ β β
β β β Speech-to-Text β β β β Sentiment β β β
β β β Transcription β β β β Analysis β β β
β β βββββββββββββββββββ β β βββββββββββββββββββ β β
β β β β βββββββββββββββββββ β β
β β β β β Companion β β β
β β β β β Responses β β β
β β β β βββββββββββββββββββ β β
β βββββββββββββββββββββββββββ βββββββββββββββββββββββββββ β
βββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββ
β
βΌ
βββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββ
β DATA MANAGEMENT LAYER β
β βββββββββββββββ βββββββββββββββ βββββββββββββββ βββββββββββββββ β
β β Session β β History β β JSON β β Trend β β
β β State β β Tracking β β Export β β Charts β β
β βββββββββββββββ βββββββββββββββ βββββββββββββββ βββββββββββββββ β
βββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββ
| Package | Version | Purpose | Why We Use It |
|---|---|---|---|
| streamlit | 1.32+ | Web Framework | Creates the interactive web interface without HTML/CSS/JavaScript. Handles buttons, sliders, and real-time updates. |
| openai | 1.107+ | AI Services | Connects to OpenAI's APIs for speech transcription (Whisper) and text analysis (GPT-3.5). The brain of the operation. |
| sounddevice | 0.5+ | Audio Recording | Captures microphone input in real-time. Provides low-latency recording with simple Python bindings to PortAudio. |
| librosa | Latest | Audio Analysis | Extracts vocal features like pitch, energy, and tempo. Essential for understanding the emotional tone in someone's voice. |
| numpy | 1.26+ | Numerical Computing | Handles audio data as arrays. Every sound wave becomes a series of numbers that need mathematical operations. |
| scipy | 1.11+ | WAV File I/O | Saves recorded audio to WAV files and reads them back. Provides reliable file format handling. |
| python-dotenv | 1.0+ | Environment Variables | Securely loads the OpenAI API key from a .env file, keeping secrets out of the source code. |
| nest-asyncio | 1.6+ | Async Support | Allows nested event loops, letting Streamlit and async OpenAI calls work together smoothly. |
# Without these, you'd need to build:
# 1. Custom audio recording system β sounddevice
# 2. Signal processing algorithms β librosa
# 3. AI models from scratch β OpenAI
# 4. Web server + frontend β Streamlit
# 5. Data structures β numpy
# 6. File handling β scipy
# 7. Security β python-dotenv- Python 3.8 or higher
- Microphone access
- OpenAI API key (Get one here)
- Clone the repository
git clone https://github.com/casss20/Audio_sentiment_analyser.git
cd Audio_sentiment_analyser- Create a virtual environment (recommended)
# Windows
python -m venv venv
venv\Scripts\activate
# Mac/Linux
python3 -m venv venv
source venv/bin/activate- Install dependencies
pip install -r requirements.txt- Set up environment variables
# Create .env file
echo "OPENAI_API_KEY=your-api-key-here" > .env- Run the application
# Version 1 (Minimal)
streamlit run app.py
# Version 2 (Luxury)
streamlit run app2.pygraph LR
A[Your Voice] --> B[Microphone]
B --> C[SoundDevice Recording]
C --> D[NumPy Array]
D --> E[Two Processing Paths]
E --> F[Path 1: Transcription]
E --> G[Path 2: Tone Analysis]
F --> H[OpenAI Whisper]
H --> I[Text]
G --> J[Librosa Features]
J --> K[Energy, Pitch, Tempo, MFCC]
I --> L[GPT-3.5 Sentiment]
K --> L
L --> M[Combined Analysis]
M --> N[Emotion Detection]
N --> O[AI Companion Response]
O --> P[Display Results]
P --> Q[Save to History]
# What each vocal feature reveals:
features = {
"energy": "Loudness - indicates excitement or anger",
"pitch_variability": "Emotional range - monotone vs expressive",
"speaking_rate": "Speed - nervousness or calmness",
"volume_variability": "Dynamic range - emotional intensity",
"mfcc_variance": "Voice texture - roughness vs smoothness",
"spectral_centroid": "Voice brightness - brightness vs darkness",
"tempo": "Rhythm - measured in BPM (beats per minute)"
}audio-sentiment-analyzer/
β
βββ π app.py # Version 1: Minimal implementation
βββ π app2.py # Version 2: Luxury implementation
βββ π requirements.txt # All dependencies
βββ π .env # Your API keys (not in repo)
βββ π .gitignore # Git ignore rules
βββ π README.md # This file
β
βββ π .streamlit/ # Streamlit configuration
β βββ π config.toml # Theme settings for Version 2
β
βββ π assets/ # Images and resources
β βββ π demo.gif
β βββ π screenshot.png
β
βββ π docs/ # Additional documentation
βββ π architecture.md
βββ π api_reference.md
/* Design principles embodied in CSS */
.design-principles {
--luxury-dark: "Deep blacks and subtle textures create sophistication";
--accent-green: "#c8f542 brings energy and life to dark surfaces";
--typography: "Bebas Neue for power, Cormorant for elegance, DM Mono for precision";
--animations: "Subtle motion guides attention without distraction";
--noise-overlay: "Analog texture in digital space adds warmth";
}Try these phrases to see the system in action:
test_cases = [
{
"phrase": "I'm so excited about my new project!",
"tone": "excited",
"expected": "excited/happy"
},
{
"phrase": "I'm fine",
"tone": "angry",
"expected": "frustrated (mismatch detected)"
},
{
"phrase": "That's wonderful",
"tone": "sarcastic",
"expected": "sarcastic"
},
{
"phrase": "I can't do this anymore",
"tone": "tired",
"expected": "tired/frustrated"
}
]Contributions are welcome! Areas for improvement:
- Add real-time emotion visualization
- Implement multiple language support
- Add voice biometrics/user recognition
- Create mobile-responsive design
- Add export to PDF reports
- Implement A/B testing framework
MIT License - feel free to use this for learning, portfolio, or commercial projects.
- OpenAI for Whisper and GPT APIs
- Librosa team for audio analysis tools
- Streamlit for the amazing web framework
- The open-source community for inspiration
Built with β€οΈ for understanding human emotion through technology
This README is living documentation - last updated February 2026
