AI Trends Knowledge Graph

Turn your customer support tickets into actionable insights with advanced AI-powered analytics.

I Built it at Petco

Overview

Transform your customer support data into a powerful knowledge graph that reveals hidden patterns, customer behaviors, and emerging trends. This system combines state-of-the-art LLMs with robust data processing to help you:

• Understand Customer Needs - Automatically categorize and tag support tickets using AI to identify what your customers really want
• Spot Emerging Issues - Detect patterns and spikes in customer issues before they become major problems
• Track Customer Journeys - Follow individual customer interactions across multiple tickets to improve their experience
• Make Data-Driven Decisions - Get clear visualizations and insights to guide your support strategy

Secure PII Handling

The system includes a robust PII (Personally Identifiable Information) handling system that ensures customer privacy and compliance:

1. Intelligent PII Detection

   • LLM-powered detection of multiple PII types:
     • Names (excluding system IDs)
     • Email addresses
     • Credit card numbers
     • Phone numbers
     • Physical addresses
   • Context-aware detection that understands natural language
   • Multi-pass scanning to catch all PII instances

2. Secure Anonymization

   • Replaces PII with unique UUIDs in format: <PII_TYPE>_uuid
   • Maintains data consistency across multiple mentions
   • Preserves relationships between anonymized entities
   • Enables analysis without exposing sensitive data

3. Secure Storage

   • Original PII data stored in encrypted format
   • Secure mapping between UUIDs and original values
   • Mapping stored in protected secure/ directory
   • Access controls and audit logging
   • Automatic cleanup of raw data files

4. Audit Trail

   • Tracks all PII replacements with timestamps
   • Maintains hash-based verification
   • Enables compliance reporting
   • Supports data subject access requests

How It Works

1. Smart Ticket Processing

   • Intelligent parsing of ticket data with robust handling of various formats
   • AI-powered tagging system that never returns empty results
   • Automatic extraction of key entities like customer IDs, products, and issues

2. Advanced Pattern Recognition

   • Real-time trend detection across multiple dimensions
   • Customer segmentation based on behavior patterns
   • Temporal analysis to identify seasonal trends and anomalies

3. Powerful Knowledge Graph

   • Custom-built graph structure optimized for support ticket analysis
   • Rich relationship mapping between customers, issues, and solutions
   • Flexible querying capabilities for deep insight extraction

4. Intelligent Analysis

   • LLM-powered trend analysis that goes beyond simple statistics
   • Automatic categorization of emerging issue types
   • Smart correlation detection between different types of issues

5. Clear Visualizations

   • Interactive dashboards showing key metrics and trends
   • Customer journey mapping and visualization
   • Temporal heat maps of issue frequencies and patterns

Why Use This?

• Proactive Support: Identify and address issues before they affect more customers
• Resource Optimization: Better understand peak times and common issues to optimize staffing
• Customer Experience: Track individual customer journeys to provide more personalized support
• Strategic Insights: Make informed decisions about product improvements and support processes
• Privacy First: Analyze customer data while maintaining strict privacy controls

Features

• AI-Powered Tag Generation

  • Multi-category tagging system:
    • Issue Type: Identifies specific problem types
    • Product: Tags related products/services
    • Customer Sentiment: Captures emotional context
    • Urgency: Indicates priority level
    • Technical: Identifies technical aspects
    • Business: Tags business processes
    • Process: Workflow-related tags
  • GBNF Grammar-constrained outputs
    • Structured tags with name, category, and confidence
    • Semantic tags with descriptive text and relevance
    • Ensures consistent JSON formatting
    • Validates outputs against defined grammar rules
  • Confidence scoring for each tag
  • Tag correlation analysis
  • Trend tracking across categories

• Knowledge Graph Analysis

  • Graph-based ticket representation
  • Relationship mapping between tickets, tags, and customers
  • Pattern detection across multiple dimensions
  • Temporal trend analysis

• Interactive Visualizations

  • Tag Correlation Network
    • Force-directed graph showing tag relationships
    • Node size indicates tag frequency
    • Edge thickness shows correlation strength
    • Hover information displays detailed metrics
    • Distinguishes between structured and semantic tags
  • Time Series Analysis
    • Monthly ticket volume trends
    • Tag usage evolution over time
    • Seasonal pattern detection
    • Day-of-week and month-of-year distributions
  • Customer Patterns
    • Customer segmentation
    • Journey analysis
    • Tag usage patterns

Pipeline Structure

The system consists of three main pipeline scripts:

1. Data Ingestion (run_data_ingest.py)

   • Parses raw ticket data from input files
   • Detects and anonymizes PII information with UUIDs
   • Generates output files:
     • tickets.json: Clean, anonymized tickets without tags
     • tickets_with_tags.json: Anonymized tickets with AI-generated tags
   • Ensures robust error handling and PII protection
   • Automatically removes raw data files after processing

2. Analysis Pipeline (run_pipeline.py)

   • Works with anonymized data to ensure privacy
   • Generates insights while maintaining PII protection
   • Creates knowledge graph from anonymized entities

3. Visualization Pipeline (run_visualizations.py)

   • Creates interactive visualizations from anonymized data
   • Supports drill-down without exposing PII
   • Enables pattern analysis while maintaining privacy

Directory Structure

ai_trends_kg/
├── scripts/
│   ├── pii_handler.py      # PII detection and anonymization
│   ├── data_ingestion.py   # Data processing pipeline
│   ├── tag_generator.py    # AI-powered tagging
│   └── ...
├── secure/                 # Protected directory for PII mappings
│   └── .gitignore         # Prevents accidental commits
├── outputs/
│   ├── tickets.json       # Anonymized tickets
│   └── visualizations/    # Generated visualizations
└── README.md

Security Best Practices

1. Data Protection

   • Raw data files are automatically deleted after processing
   • PII mappings are stored in a protected directory
   • All sensitive data is excluded from version control
   • Access controls are enforced on secure storage

2. Privacy by Design

   • Analysis works with anonymized data only
   • UUIDs are used instead of sequential IDs
   • Consistent anonymization across related entities
   • Support for data deletion requests

3. Compliance Support

   • Audit trails for all PII handling
   • Hash verification of PII mappings
   • Data subject access request support
   • Secure PII recovery when authorized

Business Impact

Operational Efficiency

• Proactive Issue Detection: Identify emerging problems before they escalate
• Resource Optimization: Better allocation of support resources based on trends
• Customer Satisfaction: Improved response times and personalized support
• Cost Reduction: Reduced support volume through pattern-based prevention

Strategic Value

• Data-Driven Decisions: Insights guide product development and process improvements
• Customer Journey Optimization: Track and improve the complete customer experience
• Trend Forecasting: Predict future support needs and resource requirements
• Compliance Assurance: Maintain privacy standards while gaining valuable insights

Technical Implementation Highlights

AI-Powered Processing

• Multi-Model Approach: Combines different LLMs for optimal results
• Grammar-Constrained Outputs: Ensures consistent and valid tag generation
• Context-Aware Analysis: Understands business context and customer intent
• Confidence Scoring: Provides reliability metrics for all AI-generated content

Scalable Architecture

• Batch Processing: Efficiently handles large volumes of ticket data
• Incremental Updates: Processes only new or changed data
• Parallel Processing: Distributes workload across multiple processors
• Memory Optimization: Handles large datasets without excessive memory usage

Visualization Excellence

• Interactive Dashboards: Drill-down capabilities for detailed analysis
• Real-time Updates: Live data refresh for current insights
• Export Capabilities: Generate reports and share insights
• Mobile Responsive: Access insights on any device

This system represents a comprehensive solution for transforming customer support data into strategic business intelligence while maintaining the highest standards of privacy and security.