Color Extraction Guide
Learn how VideoIntel extracts dominant colors from videos using K-means clustering, smart sampling, and optimized pixel analysis for fast and accurate results.
Overview
Color extraction identifies the most dominant colors in a video by analyzing frames and grouping similar colors together. This is useful for:
- UI Theming: Generate color schemes matching video content
- Video Categorization: Classify videos by color palette
- Design Systems: Create brand-aligned color palettes
- Thumbnail Matching: Ensure UI colors complement video colors
- Mood Analysis: Understand emotional tone through color
// Extract 5 dominant colors from a video
const colors = await videoIntel.extractColors(video, {
count: 5,
sampleFrames: 10,
quality: 'balanced'
});
// Use the colors
colors.forEach(color => {
console.log(`${color.hex} - ${color.percentage}% of video`);
// Output example: "#3B82F6 - 32.5% of video"
});How It Works
VideoIntel's color extraction uses a sophisticated multi-stage pipeline:
Step 1: Frame Sampling
Extract frames at evenly distributed intervals throughout the video. Default: 10 frames from start to end, ensuring representative sampling of the entire video.
Step 2: Pixel Extraction
Extract pixel data from each frame and sample based on quality setting. Balanced quality samples 30% of pixels for good accuracy with fast processing.
Step 3: K-Means Clustering
Group similar colors together using K-means clustering algorithm. This finds the N most dominant colors by iteratively improving cluster centers.
Step 4: Color Conversion
Convert cluster centers to multiple color formats (hex, RGB, HSL) and calculate dominance percentages for each color.
K-Means Clustering Algorithm
K-means is an unsupervised machine learning algorithm that groups similar colors:
// Simplified K-means pseudo-code
function kMeansClustering(pixels: RGB[], k: number) {
// 1. Initialize K random cluster centers (K-means++ for better results)
let centers = initializeKMeansPlusPlus(pixels, k);
for (let iteration = 0; iteration < maxIterations; iteration++) {
// 2. Assign each pixel to nearest cluster center
const assignments = pixels.map(pixel =>
findNearestCenter(pixel, centers)
);
// 3. Recalculate centers as mean of assigned pixels
const newCenters = calculateClusterMeans(pixels, assignments, k);
// 4. Check convergence (centers stop moving)
if (hasConverged(centers, newCenters)) {
break;
}
centers = newCenters;
}
// 5. Calculate dominance percentage for each cluster
const percentages = calculatePercentages(assignments, k);
return centers.map((center, i) => ({
color: center,
percentage: percentages[i]
}));
}K-Means++ Initialization
VideoIntel uses K-means++ for smarter initial cluster placement, which:
- Spreads initial centers far apart in color space
- Converges 2-3x faster than random initialization
- Produces more consistent results across runs
- Avoids poor local minima
// K-means++ initialization
function initializeKMeansPlusPlus(pixels: RGB[], k: number) {
const centers: RGB[] = [];
// 1. Choose first center randomly
centers.push(pixels[randomIndex()]);
// 2. Choose remaining centers based on distance
for (let i = 1; i < k; i++) {
// Calculate distance from each pixel to nearest existing center
const distances = pixels.map(pixel => {
const minDist = Math.min(...centers.map(c =>
colorDistance(pixel, c)
));
return minDist * minDist; // Square for probability weighting
});
// 3. Choose next center with probability proportional to distance
const nextCenter = weightedRandomChoice(pixels, distances);
centers.push(nextCenter);
}
return centers;
}Frame Sampling Strategy
VideoIntel samples frames evenly throughout the video to ensure representative color coverage:
// Example: 60-second video, 10 frames
const interval = 60 / 10; // 6 seconds
const timestamps = [3, 9, 15, 21, 27, 33, 39, 45, 51, 57];
// ↑ Start at 0.5 × interval to avoid black frames
// Distribution ensures:
// ✓ Start, middle, and end are all represented
// ✓ No clustering in one section
// ✓ Captures color changes throughout videoPixel Extraction & Sampling
Analyzing every pixel would be slow. VideoIntel uses adaptive sampling based on quality:
| Quality | Sampling Rate | Pixels Analyzed | Processing Time |
|---|---|---|---|
| fast | 10% | ~200K pixels | 1-2 seconds |
| balanced ⭐ | 30% | ~600K pixels | 3-5 seconds |
| best | 100% | ~2M pixels | 10-15 seconds |
// Pixel extraction with sampling
function extractPixels(frame: Canvas, samplingRate: number) {
const ctx = frame.getContext('2d');
const imageData = ctx.getImageData(0, 0, frame.width, frame.height);
const pixels: RGB[] = [];
// Calculate step size based on sampling rate
// 30% sampling = step every ~3 pixels
const step = Math.round(1 / samplingRate);
for (let i = 0; i < imageData.data.length; i += 4 * step) {
const r = imageData.data[i];
const g = imageData.data[i + 1];
const b = imageData.data[i + 2];
const a = imageData.data[i + 3];
// Skip fully transparent pixels
if (a > 0) {
pixels.push([r, g, b]);
}
}
return pixels;
}Color Clustering Process
The clustering process iteratively refines color groups:
// Example clustering progression for 5 colors
Iteration 0 (Initial):
Centers: [Random colors from K-means++ initialization]
Iteration 1:
Centers: [Updated based on pixel assignments]
Movement: Large (centers adjust to data)
Iteration 5:
Centers: [Refined positions]
Movement: Moderate (converging)
Iteration 10:
Centers: [Nearly final positions]
Movement: Small (<1% change)
Iteration 12:
Centers: [Converged]
Movement: Negligible → STOP
Result:
Color 1: #2563EB (Blue) - 35.2%
Color 2: #10B981 (Green) - 28.7%
Color 3: #F59E0B (Orange) - 18.3%
Color 4: #6B7280 (Gray) - 12.1%
Color 5: #1F2937 (Dark) - 5.7%Configuration Options
Fine-tune color extraction for your specific needs:
const colors = await videoIntel.extractColors(video, {
// Number of colors to extract (2-10)
count: 5, // Default: 5
// Number of frames to sample from video
sampleFrames: 10, // Default: 10
// Sampling quality
quality: 'balanced', // 'fast' | 'balanced' | 'best'
});
// Output format
interface Color {
hex: string; // "#3B82F6"
rgb: [number, number, number]; // [59, 130, 246]
hsl: [number, number, number]; // [217, 91, 60]
percentage: number; // 32.5 (percentage of video)
}Choosing Color Count
The optimal number of colors depends on your use case:
// Minimalist palette (2-3 colors)
const minimal = await videoIntel.extractColors(video, { count: 3 });
// Use case: Simple color scheme, accent colors
// Standard palette (4-5 colors)
const standard = await videoIntel.extractColors(video, { count: 5 });
// Use case: UI theming, brand colors
// Rich palette (6-8 colors)
const rich = await videoIntel.extractColors(video, { count: 8 });
// Use case: Detailed analysis, design systems
// Comprehensive (9-10 colors)
const comprehensive = await videoIntel.extractColors(video, { count: 10 });
// Use case: Color grading, mood boardsBest Practices
1. Match Quality to Use Case
// Real-time preview (prioritize speed)
const previewColors = await videoIntel.extractColors(video, {
count: 3,
sampleFrames: 5,
quality: 'fast'
});
// Production color palette (balance speed & accuracy)
const productionColors = await videoIntel.extractColors(video, {
count: 5,
sampleFrames: 10,
quality: 'balanced' // ⭐ Recommended
});
// High-precision analysis (prioritize accuracy)
const precisionColors = await videoIntel.extractColors(video, {
count: 8,
sampleFrames: 20,
quality: 'best'
});2. Filter Out Near-White/Black (Optional)
For UI theming, you might want to exclude neutral colors:
function filterNeutrals(colors: Color[], threshold = 15) {
return colors.filter(color => {
const [r, g, b] = color.rgb;
const max = Math.max(r, g, b);
const min = Math.min(r, g, b);
// Skip near-white colors (high brightness, low saturation)
if (max > 240 && (max - min) < threshold) {
return false;
}
// Skip near-black colors (low brightness)
if (max < 20) {
return false;
}
return true;
});
}
// Usage
const allColors = await videoIntel.extractColors(video, { count: 8 });
const vibrantColors = filterNeutrals(allColors);3. Sort by Dominance or Brightness
const colors = await videoIntel.extractColors(video, { count: 5 });
// Already sorted by dominance (most common first)
const mostDominant = colors[0];
// Sort by brightness (for gradients)
const byBrightness = [...colors].sort((a, b) => {
const brightnessA = (a.rgb[0] + a.rgb[1] + a.rgb[2]) / 3;
const brightnessB = (b.rgb[0] + b.rgb[1] + b.rgb[2]) / 3;
return brightnessA - brightnessB; // Dark to light
});
// Sort by saturation (most vibrant first)
const bySaturation = [...colors].sort((a, b) => {
return b.hsl[1] - a.hsl[1]; // High to low saturation
});Performance Optimization
Benchmarks
| Configuration | Pixels Analyzed | Time (720p) | Accuracy |
|---|---|---|---|
| 5 frames, fast | ~100K | 0.5-1s | 85% |
| 10 frames, balanced | ~600K | 3-5s | 95% |
| 20 frames, best | ~4M | 12-18s | 98% |
💡 Performance Tip
The 'balanced' quality setting provides 95% accuracy at 5x the speed of 'best' quality. Unless you need pixel-perfect precision, balanced is the recommended choice.
Common Use Cases
UI Theming
async function generateTheme(videoFile: File) {
const colors = await videoIntel.extractColors(videoFile, {
count: 5,
quality: 'balanced'
});
// Sort by dominance
const [primary, secondary, accent, ...rest] = colors;
return {
primary: primary.hex, // Most dominant color
secondary: secondary.hex, // Second most dominant
accent: accent.hex, // Third for highlights
// Create variations
primaryLight: lighten(primary.hex, 20),
primaryDark: darken(primary.hex, 20),
// Use in CSS
css: `
--color-primary: ${primary.hex};
--color-secondary: ${secondary.hex};
--color-accent: ${accent.hex};
`
};
}Color Palette Generator
async function createPalette(videoFile: File) {
const colors = await videoIntel.extractColors(videoFile, {
count: 8,
quality: 'best'
});
// Generate palette UI
const palette = document.createElement('div');
palette.className = 'color-palette';
colors.forEach(color => {
const swatch = document.createElement('div');
swatch.className = 'color-swatch';
swatch.style.backgroundColor = color.hex;
swatch.style.width = `${color.percentage}%`;
swatch.title = `${color.hex} - ${color.percentage}%`;
// Add color info
const info = document.createElement('div');
info.innerHTML = `
<strong>${color.hex}</strong>
<span>RGB(${color.rgb.join(', ')})</span>
<span>HSL(${color.hsl.join(', ')})</span>
<span>${color.percentage}%</span>
`;
swatch.appendChild(info);
palette.appendChild(swatch);
});
return palette;
}Video Mood Analysis
async function analyzeVideoMood(videoFile: File) {
const colors = await videoIntel.extractColors(videoFile, {
count: 5,
quality: 'balanced'
});
// Analyze color characteristics
const avgBrightness = colors.reduce((sum, c) =>
sum + (c.rgb[0] + c.rgb[1] + c.rgb[2]) / 3, 0
) / colors.length;
const avgSaturation = colors.reduce((sum, c) =>
sum + c.hsl[1], 0
) / colors.length;
// Determine mood
let mood = 'neutral';
if (avgBrightness > 180 && avgSaturation > 50) {
mood = 'vibrant'; // Bright and saturated
} else if (avgBrightness < 100) {
mood = 'dark'; // Low brightness
} else if (avgSaturation < 20) {
mood = 'muted'; // Low saturation
} else if (avgBrightness > 150) {
mood = 'light'; // High brightness
}
return {
mood,
brightness: avgBrightness,
saturation: avgSaturation,
colors: colors.map(c => c.hex),
};
}Gradient Generation
async function createGradient(videoFile: File) {
const colors = await videoIntel.extractColors(videoFile, {
count: 5,
quality: 'balanced'
});
// Sort by brightness for smooth gradient
const sorted = [...colors].sort((a, b) => {
const brightnessA = (a.rgb[0] + a.rgb[1] + a.rgb[2]) / 3;
const brightnessB = (b.rgb[0] + b.rgb[1] + b.rgb[2]) / 3;
return brightnessA - brightnessB;
});
// Create CSS gradient
const stops = sorted.map((color, i) =>
`${color.hex} ${(i / (sorted.length - 1)) * 100}%`
).join(', ');
return {
linear: `linear-gradient(to right, ${stops})`,
radial: `radial-gradient(circle, ${stops})`,
colors: sorted.map(c => c.hex),
};
}🚀 Next Steps
- • Learn about Thumbnail Generation
- • Explore Scene Detection
- • Try the Interactive Playground
- • View Complete API Reference