Troubleshooting - YouTube Automation Agent

Overview

This guide helps you identify and resolve common issues. The system includes comprehensive logging and error handling based on patterns from utils/logger.js and error handling throughout the codebase.

Logging System

Understanding Log Levels

The system uses Winston logging with multiple levels (from utils/logger.js:14-20):

// Set log level in .env
LOG_LEVEL=info  // Options: error, warn, info, debug

ERROR - Critical failures

System errors that prevent normal operation. Always logged to logs/error.log.

[ERROR] [ContentStrategy] Failed to fetch YouTube trends: API quota exceeded

WARN - Potential issues

Non-critical issues that should be monitored.

[WARN] [ScriptWriter] No historical data found, starting fresh

INFO - Normal operations

Standard operational messages.

[INFO] [MainAgent] ✓ strategy agent initialized

DEBUG - Detailed diagnostics

Detailed information for troubleshooting. Only shown in development.

[DEBUG] [Database] Executing query: SELECT * FROM productions

Log Files Location

Logs are stored in separate files (from utils/logger.js:23-43):

logs/
├── combined.log          # All logs (max 5MB × 5 files)
├── error.log            # Errors only (max 5MB × 3 files)
├── mainalent.log        # Main agent logs
├── contentstrategy.log  # Strategy agent logs
├── scriptwriter.log     # Script writer logs
└── [component].log      # Component-specific logs

Viewing Logs

# View all recent logs
tail -f logs/combined.log

# View errors only
tail -f logs/error.log

# View specific agent logs
tail -f logs/contentstrategy.log

# Search for specific errors
grep "Failed" logs/combined.log

# View last 100 lines with context
tail -n 100 logs/combined.log

Common Issues

Authentication Failures

YouTube API Authentication Error

Symptoms:

[ERROR] [CredentialManager] YouTube credentials not configured
[ERROR] Failed to initialize: YouTube authentication required

Solutions:

Verify credentials file exists

ls -la config/credentials.json
ls -la config/tokens.json

Check credentials format

Ensure config/credentials.json matches the example from config/credentials.example.json:1-32:

{
  "youtube": {
    "client_id": "YOUR_CLIENT_ID.apps.googleusercontent.com",
    "client_secret": "YOUR_CLIENT_SECRET",
    "redirect_uris": ["urn:ietf:wg:oauth:2.0:oob"]
  },
  "openai": {
    "apiKey": "sk-...",
    "model": "gpt-4-turbo-preview"
  }
}

Re-authenticate

npm run credentials:setup

Follow the prompts to re-authenticate with YouTube.

Verify API is enabled

Go to Google Cloud Console
Enable YouTube Data API v3
Enable YouTube Analytics API

Token Expired

Symptoms:

[ERROR] Invalid Credentials: Request had invalid authentication credentials

Solution: Tokens automatically refresh, but if they’re corrupted:

# Delete old tokens
rm config/tokens.json

# Re-authenticate
node utils/credential-manager.js setup

Database Issues

Database Locked

Symptoms:

[ERROR] [Database] SQLITE_BUSY: database is locked

Solutions:

Check for multiple instances:

# Check if agent is already running
ps aux | grep "node index.js"

# Kill existing processes
kill [PID]

Close database connections:

// Properly close database before restart
const agent = new YouTubeAutomationAgent();
await agent.db.close();

Database corruption:

# Check database integrity
sqlite3 data/youtube_automation.db "PRAGMA integrity_check;"

# If corrupted, restore from backup
cp data/backup_[timestamp].db data/youtube_automation.db

Missing Tables

Symptoms:

[ERROR] no such table: content_strategies

Solution: Reinitialize database (from database/db.js:33-195):

# Delete and recreate
rm data/youtube_automation.db
node index.js

# Or manually initialize
node -e "const {Database} = require('./database/db'); const db = new Database(); db.initialize();"

Content Generation Failures

Script Generation Timeout

Symptoms:

[ERROR] [ScriptWriter] Failed to generate script: Request timeout

Solutions:

Increase timeout in environment:

.env

OPENAI_TIMEOUT=120000  # 2 minutes

Check OpenAI API status:

curl https://status.openai.com/api/v2/status.json

Implement retry logic:

const RETRY_ATTEMPTS = 3;
const RETRY_DELAY = 5000;

for (let attempt = 1; attempt <= RETRY_ATTEMPTS; attempt++) {
  try {
    const script = await this.agents.scriptWriter.generateScript(strategy);
    break;
  } catch (error) {
    if (attempt === RETRY_ATTEMPTS) throw error;
    await new Promise(resolve => setTimeout(resolve, RETRY_DELAY));
    this.logger.warn(`Retry attempt ${attempt}/${RETRY_ATTEMPTS}`);
  }
}

AI Model Quota Exceeded

Symptoms:

[ERROR] OpenAI API error: Rate limit exceeded

Solutions:

Check your OpenAI quota

Visit https://platform.openai.com/account/usage

Implement rate limiting

const delay = ms => new Promise(resolve => setTimeout(resolve, ms));

// Add delay between API calls
await delay(2000); // 2 second delay

Switch to cheaper model temporarily

config/credentials.json

{
  "openai": {
    "apiKey": "sk-...",
    "model": "gpt-3.5-turbo"  // Instead of gpt-4
  }
}

Publishing Issues

Upload Failed

Symptoms:

[ERROR] [PublishingScheduling] Failed to upload video: Quota exceeded

YouTube API Quota Information:

Daily quota: 10,000 units
Video upload: ~1,600 units
Video update: 50 units
Analytics: 1 unit per request

Solutions:

Monitor quota usage:

// Add quota tracking
async trackQuotaUsage(operation, cost) {
  const today = new Date().toISOString().split('T')[0];
  const used = await this.db.getSetting(`quota_${today}`) || 0;
  const newUsed = parseInt(used) + cost;
  
  await this.db.setSetting(`quota_${today}`, newUsed.toString());
  
  if (newUsed > 9000) {
    this.logger.warn(`Approaching quota limit: ${newUsed}/10,000`);
  }
}

Implement queue system:

// Spread uploads throughout the day
if (quotaRemaining < 1600) {
  await this.db.updateScheduleEntry({
    ...entry,
    status: 'paused',
    error: 'Quota limit - will retry tomorrow'
  });
}

Video Processing Stuck

Symptoms:

[WARN] Video uploaded but still processing after 30 minutes

Solution:

// Implement processing check
async checkVideoProcessingStatus(videoId) {
  const youtube = this.credentials.getYouTubeClient();
  
  const response = await youtube.videos.list({
    part: 'status,processingDetails',
    id: videoId
  });
  
  const video = response.data.items[0];
  
  if (video.status.uploadStatus === 'processed') {
    return 'complete';
  } else if (video.status.uploadStatus === 'failed') {
    throw new Error(`Processing failed: ${video.status.failureReason}`);
  }
  
  return 'processing';
}

Scheduler Issues

Cron Jobs Not Running

Symptoms:

[INFO] Scheduled task created but content not generating

Debug steps:

schedules/daily-automation.js

// Add debugging to scheduler
async setupScheduledTasks() {
  this.scheduledTasks.set('daily-content-generation', 
    cron.schedule('0 6 * * *', async () => {
      console.log('CRON TRIGGERED:', new Date().toISOString());
      
      if (this.isEnabled) {
        console.log('Running daily generation...');
        await this.runDailyContentGeneration();
      } else {
        console.log('Automation disabled, skipping');
      }
    }, { 
      scheduled: true,
      timezone: 'America/New_York' // Set your timezone
    })
  );
  
  // Verify task is scheduled
  this.scheduledTasks.forEach((task, name) => {
    console.log(`Task ${name}:`, task.options);
  });
}

Test scheduler:

// Test with shorter interval
cron.schedule('*/1 * * * *', async () => {
  console.log('Test task running every minute');
});

Performance Issues

High Memory Usage

Symptoms:

[WARN] System health score: 65/100
memory: { heapUsed: 950MB, heapTotal: 1024MB }

Solutions:

Monitor memory:

const logMemoryUsage = () => {
  const used = process.memoryUsage();
  console.log('Memory Usage:');
  for (let key in used) {
    console.log(`  ${key}: ${Math.round(used[key] / 1024 / 1024 * 100) / 100} MB`);
  }
};

setInterval(logMemoryUsage, 60000); // Every minute

Clear caches:

// Clear agent caches periodically
async clearCaches() {
  this.agents.strategy.trendingTopics = [];
  this.agents.strategy.competitorData = [];
  
  if (global.gc) {
    global.gc();
    this.logger.info('Garbage collection triggered');
  }
}

Limit concurrent operations:

// Process in batches instead of all at once
const batchSize = 5;
for (let i = 0; i < items.length; i += batchSize) {
  const batch = items.slice(i, i + batchSize);
  await Promise.all(batch.map(item => processItem(item)));
  await new Promise(resolve => setTimeout(resolve, 1000)); // Pause between batches
}

Slow Response Times

Debug with performance logging:

// Use built-in timer (from logger.js:104-113)
const timer = this.logger.startTimer('Content Generation Pipeline');

const strategy = await this.agents.strategy.generateContentStrategy();
const t1 = this.logger.startTimer('Strategy');
t1.end(); // Logs: "Strategy completed in Xms"

const script = await this.agents.scriptWriter.generateScript(strategy);
const t2 = this.logger.startTimer('Script');
t2.end();

timer.end(); // Logs total pipeline time

Error Recovery

Automatic Retry Pattern

Implement throughout your code:

utils/retry-helper.js

class RetryHelper {
  static async withRetry(fn, options = {}) {
    const {
      maxAttempts = 3,
      delayMs = 5000,
      backoff = true,
      onRetry = null
    } = options;
    
    let lastError;
    
    for (let attempt = 1; attempt <= maxAttempts; attempt++) {
      try {
        return await fn();
      } catch (error) {
        lastError = error;
        
        if (attempt === maxAttempts) {
          throw error;
        }
        
        const delay = backoff ? delayMs * attempt : delayMs;
        
        if (onRetry) {
          onRetry(attempt, maxAttempts, error);
        }
        
        await new Promise(resolve => setTimeout(resolve, delay));
      }
    }
    
    throw lastError;
  }
}

// Usage:
const result = await RetryHelper.withRetry(
  () => this.agents.strategy.generateContentStrategy(),
  {
    maxAttempts: 3,
    delayMs: 5000,
    backoff: true,
    onRetry: (attempt, max, error) => {
      this.logger.warn(`Retry ${attempt}/${max}: ${error.message}`);
    }
  }
);

Graceful Degradation

// Fallback to simpler operations if primary fails
async generateContentWithFallback(topic) {
  try {
    // Try with GPT-4
    return await this.generateWithModel('gpt-4-turbo-preview', topic);
  } catch (error) {
    this.logger.warn('GPT-4 failed, falling back to GPT-3.5');
    
    try {
      // Fallback to GPT-3.5
      return await this.generateWithModel('gpt-3.5-turbo', topic);
    } catch (fallbackError) {
      this.logger.error('All models failed, using template');
      // Ultimate fallback to template
      return this.generateFromTemplate(topic);
    }
  }
}

Health Monitoring

System Health Check

The automation includes health monitoring (from schedules/daily-automation.js:476-526):

// Access health endpoint
curl http://localhost:3456/health

// Response:
{
  "status": "healthy",
  "initialized": true,
  "agents": ["strategy", "scriptWriter", ...],
  "timestamp": "2026-03-05T10:30:00.000Z"
}

Custom Health Checks

async performDetailedHealthCheck() {
  const health = {
    database: await this.checkDatabase(),
    youtube: await this.checkYouTubeAPI(),
    openai: await this.checkOpenAI(),
    disk: await this.checkDiskSpace(),
    memory: this.checkMemory()
  };
  
  const issues = Object.entries(health)
    .filter(([, status]) => !status.healthy)
    .map(([component, status]) => `${component}: ${status.error}`);
  
  if (issues.length > 0) {
    this.logger.error('Health check failed:', issues);
    // Send alert notification
    await this.sendAlert('System Health Issues', issues.join('\n'));
  }
  
  return health;
}

async checkDatabase() {
  try {
    await this.db.getAllRows('SELECT 1');
    return { healthy: true };
  } catch (error) {
    return { healthy: false, error: error.message };
  }
}

async checkYouTubeAPI() {
  try {
    const youtube = this.credentials.getYouTubeClient();
    await youtube.channels.list({ part: 'snippet', mine: true });
    return { healthy: true };
  } catch (error) {
    return { healthy: false, error: error.message };
  }
}

async checkDiskSpace() {
  const { execSync } = require('child_process');
  const output = execSync('df -h /').toString();
  const usage = parseInt(output.split('\n')[1].split(/\s+/)[4]);
  
  return {
    healthy: usage < 90,
    usage: `${usage}%`,
    error: usage >= 90 ? 'Disk space critical' : null
  };
}

Critical Issues Requiring Immediate Attention:

Database corruption
API authentication failures
Disk space > 90% full
Memory usage > 90%
Continuous failed content generation
YouTube quota consistently exceeded

Getting Help

Collecting Diagnostic Information

When reporting issues:

# Collect system info
node -v
npm -v
uname -a

# Check package versions
cat package.json | grep version

# Recent errors
tail -n 50 logs/error.log

# Database stats
sqlite3 data/youtube_automation.db "SELECT name, value FROM settings;"

# Disk usage
du -sh data/ logs/

Debug Mode

Run in debug mode:

LOG_LEVEL=debug NODE_ENV=development node index.js

Customization

Customize and extend the system

Configuration

Review configuration options

​Overview

​Logging System

​Understanding Log Levels

​Log Files Location

​Viewing Logs

​Common Issues

​Authentication Failures

​YouTube API Authentication Error

​Token Expired

​Database Issues

​Database Locked

​Missing Tables

​Content Generation Failures

​Script Generation Timeout

​AI Model Quota Exceeded

​Publishing Issues

​Upload Failed

​Video Processing Stuck

​Scheduler Issues

​Cron Jobs Not Running

​Performance Issues

​High Memory Usage

​Slow Response Times

​Error Recovery

​Automatic Retry Pattern

​Graceful Degradation

​Health Monitoring

​System Health Check

​Custom Health Checks

​Getting Help

​Collecting Diagnostic Information

​Debug Mode

Customization

Configuration

Overview

Logging System

Understanding Log Levels

Log Files Location

Viewing Logs

Common Issues

Authentication Failures

YouTube API Authentication Error

Token Expired

Database Issues

Database Locked

Missing Tables

Content Generation Failures

Script Generation Timeout

AI Model Quota Exceeded

Publishing Issues

Upload Failed

Video Processing Stuck

Scheduler Issues

Cron Jobs Not Running

Performance Issues

High Memory Usage

Slow Response Times

Error Recovery

Automatic Retry Pattern

Graceful Degradation

Health Monitoring

System Health Check

Custom Health Checks

Getting Help

Collecting Diagnostic Information

Debug Mode