linkscout-backend / COMPLETE_GUIDE.md
zpsajst's picture
Initial commit with environment variables for API keys
2398be6

πŸŽ‰ LINKSCOUT - COMBINED EXTENSION COMPLETE!

Smart Analysis. Simple Answers.


βœ… WHAT WAS DONE

I have successfully combined both extensions (mis and mis_2) into a single powerful extension called LinkScout. Here's what was created:

πŸ“‚ New Directory Structure

d:\mis_2\LinkScout\
β”œβ”€β”€ combined_server.py           ⭐ Combined backend server
β”œβ”€β”€ extension\                   πŸ“¦ Chrome extension files
β”‚   β”œβ”€β”€ manifest.json           ⭐ Extension manifest (renamed to LinkScout)
β”‚   β”œβ”€β”€ popup.html              ⭐ Combined popup UI
β”‚   β”œβ”€β”€ popup.js                ⭐ Combined popup logic
β”‚   β”œβ”€β”€ content.js              ⭐ Combined content script with highlighting + sidebar
β”‚   β”œβ”€β”€ background.js           ⭐ Service worker
β”‚   β”œβ”€β”€ styles.css              ⭐ Styling
β”‚   β”œβ”€β”€ utils\                  πŸ“ Utility scripts
β”‚   β”‚   β”œβ”€β”€ contentExtractor_v2.js
β”‚   β”‚   β”œβ”€β”€ cache.js
β”‚   β”‚   β”œβ”€β”€ chunkAnalyzer.js
β”‚   β”‚   └── contentExtractor.js
β”‚   └── icons\                  🎨 Extension icons
β”‚       β”œβ”€β”€ icon16.png
β”‚       β”œβ”€β”€ icon48.png
β”‚       └── icon128.png
β”œβ”€β”€ reinforcement_learning.py    πŸ€– RL agent (from mis)
β”œβ”€β”€ image_analysis.py           πŸ–ΌοΈ Image analysis (from mis)
β”œβ”€β”€ linguistic_fingerprint.py   πŸ” Phase 1 detection (from mis)
β”œβ”€β”€ claim_verifier.py           βœ… Phase 1 detection (from mis)
β”œβ”€β”€ source_credibility.py       ⭐ Phase 1 detection (from mis)
β”œβ”€β”€ propaganda_detector.py      πŸ“’ Phase 2 detection (from mis)
β”œβ”€β”€ entity_verifier.py          πŸ‘€ Phase 2 detection (from mis)
β”œβ”€β”€ contradiction_detector.py   ⚠️ Phase 3 detection (from mis)
β”œβ”€β”€ network_analyzer.py         🌐 Phase 3 detection (from mis)
β”œβ”€β”€ known_false_claims.py       πŸ“Š Offline database (from mis)
β”œβ”€β”€ google_search.py            πŸ”— Google API integration (from mis_2)
β”œβ”€β”€ google_config.json          πŸ”§ Google API config (from mis_2)
β”œβ”€β”€ requirements.txt            πŸ“‹ All dependencies
β”œβ”€β”€ START_SERVER.bat            πŸš€ Windows startup script
β”œβ”€β”€ START_SERVER.ps1            πŸš€ PowerShell startup script
└── README.md                   πŸ“– Complete documentation

🎯 COMBINED FEATURES

From MIS Extension (Groq-based):

βœ… Groq AI agentic analysis with 3 specialized agents βœ… Reinforcement Learning (learns from feedback) βœ… Image analysis (AI-generated image detection) βœ… Phase 1 Revolutionary Detection (Linguistic, Claims, Sources) βœ… Phase 2 Revolutionary Detection (Entities, Propaganda, Verification) βœ… Phase 3 Revolutionary Detection (Contradictions, Network patterns) βœ… Offline false claims database βœ… Web research with DuckDuckGo βœ… Color-coded highlighting

From MIS_2 Extension (Pre-trained models):

βœ… Pre-trained Models (RoBERTa, Emotion, NER, Hate Speech, Clickbait, Bias) βœ… Chunk-based paragraph analysis βœ… Google Search API integration βœ… Sidebar display (non-intrusive analysis results) βœ… Category detection (25+ news categories) βœ… Multi-language support (Hindi, Marathi, etc.) βœ… Content extraction utilities

New Combined Features:

βœ… Unified backend server combining all analysis methods βœ… Clean, modern UI with tabs (Overview, Details, Sources) βœ… Intelligent score calculation using all models βœ… Organized sidebar with percentage display and statistics βœ… Smart highlighting with severity levels (red/yellow/blue) βœ… One-click page scanning βœ… Background analysis capability βœ… Comprehensive error handling


πŸš€ HOW TO USE

Step 1: Start the Server

Option A: Using Batch Script (Recommended)

1. Navigate to d:\mis_2\LinkScout\
2. Double-click START_SERVER.bat
3. Wait for server to start (will show "Server: http://localhost:5000")

Option B: Using PowerShell Script

1. Right-click START_SERVER.ps1
2. Select "Run with PowerShell"

Option C: Manual Start

cd d:\mis_2\LinkScout
python combined_server.py

Step 2: Load the Extension

  1. Open Chrome/Edge
  2. Go to chrome://extensions or edge://extensions
  3. Enable "Developer mode" (toggle in top-right)
  4. Click "Load unpacked"
  5. Select folder: d:\mis_2\LinkScout\extension
  6. Extension should appear with πŸ” icon
  7. Pin it to toolbar for easy access

Step 3: Test It!

Test 1: Scan a News Page

  1. Open any news article (e.g., BBC, CNN, Times of India)
  2. Click the LinkScout icon
  3. Click "Scan Page" button
  4. Wait for analysis (10-30 seconds)
  5. Check results in popup
  6. Look for highlighted paragraphs on the page
  7. Click "Highlight" to see color-coded suspicious content

Test 2: Analyze Text

  1. Copy any article text
  2. Click LinkScout icon
  3. Paste text in input box
  4. Click "Analyze" button
  5. View comprehensive results

Test 3: Check URL

  1. Copy a news article URL
  2. Click LinkScout icon
  3. Paste URL in input box
  4. Click "Analyze" button
  5. Review analysis

🎨 WHAT THE EXTENSION LOOKS LIKE

Popup Interface:

  • Header: LinkScout logo with tagline "Smart Analysis. Simple Answers."
  • Input Box: Paste text or URL
  • Buttons:
    • πŸ”¬ Analyze (for text/URL)
    • πŸ“„ Scan Page (analyze current page)
    • 🎨 Highlight (show suspicious paragraphs)
    • ❌ Clear (remove highlights)
  • Results Area: Shows percentage score, verdict, and detailed analysis in tabs

Page Highlighting:

  • πŸ”΄ Red border: High risk (>70% suspicious)
  • 🟑 Yellow border: Medium risk (40-70% suspicious)
  • πŸ”΅ Blue border: Low risk (<40% suspicious)

Sidebar:

  • Appears on right side of page
  • Shows overall score and verdict
  • Lists all suspicious paragraphs
  • Includes summary and statistics
  • Can be closed with X button

πŸ”§ BACKEND FUNCTIONALITY

What the Server Does:

  1. Content Analysis Pipeline:

    Input Content
    ↓
    Pre-trained Models (RoBERTa, Emotion, NER, etc.)
    ↓
    Groq AI Analysis (Research, Analysis, Conclusion agents)
    ↓
    Revolutionary Detection (Phases 1-3)
    ↓
    Google Search Verification
    ↓
    Scoring & Verdict Calculation
    ↓
    RL Agent Suggestion (optional)
    ↓
    JSON Response with all results
    
  2. Models Used:

    • RoBERTa Fake News Classifier
    • Emotion Classifier (7 emotions)
    • Named Entity Recognition (NER)
    • Hate Speech Detector
    • Clickbait Detector
    • Bias Detector
    • Custom trained model (if available)
  3. API Endpoints:

    • POST /api/v1/analyze-chunks - Main analysis endpoint (unified)
    • POST /api/v1/analyze - Legacy endpoint
    • GET /health - Server health check

⚠️ IMPORTANT NOTES

Backend Functionality NOT Changed:

βœ… All backend modules work exactly as before βœ… No changes to analysis algorithms βœ… No changes to model loading βœ… No changes to scoring logic βœ… Both systems' functionality fully preserved

What WAS Changed:

  • Combined both servers into one unified server
  • Created new unified frontend interface
  • Merged content scripts for highlighting + sidebar
  • Added better error handling
  • Improved UI organization
  • Made popup cleaner and more modern

API Keys:

  • Groq API: Already configured in combined_server.py
  • Google API: Update google_config.json with your keys (optional)

πŸ› TROUBLESHOOTING

Server Won't Start:

Error: Port 5000 already in use
Solution: Kill existing process or change port in combined_server.py

Extension Not Loading:

Error: Manifest error
Solution: Check manifest.json for syntax errors

Models Not Loading:

Error: Model not found
Solution: Ensure D:\huggingface_cache exists and has models
Run: python -c "from transformers import AutoTokenizer; AutoTokenizer.from_pretrained('hamzab/roberta-fake-news-classification')"

No Analysis Results:

Error: Server connection failed
Solution: 
1. Check server is running (http://localhost:5000/health)
2. Check browser console for errors (F12 β†’ Console)
3. Try reloading extension

πŸ“‹ NEXT STEPS

  1. Start the Server: Run START_SERVER.bat
  2. Load Extension: Load d:\mis_2\LinkScout\extension in Chrome
  3. Test It: Visit a news site and click "Scan Page"
  4. Enjoy: Your combined extension is ready! πŸŽ‰

πŸ’‘ WHICH EXTENSION TO LOAD?

ANSWER: Load the NEW LinkScout extension!

Location: d:\mis_2\LinkScout\extension

This is the combined extension that includes ALL features from both:

  • βœ… mis extension (Groq AI, RL, Image Analysis, Revolutionary Detection)
  • βœ… mis_2 extension (Pre-trained Models, Chunk Analysis, Google Search, Sidebar)

You can now UNLOAD the old extensions (mis and mis_2) and use only LinkScout.


🎊 SUMMARY

You now have a single, powerful extension called LinkScout that:

  • Combines Groq AI + Pre-trained Models
  • Has all 8 phases of revolutionary detection
  • Uses RL to learn and improve
  • Provides clean, organized results
  • Highlights suspicious content
  • Shows analysis in sidebar
  • Integrates Google search
  • Maintains ALL backend functionality from both systems

Name: LinkScout Tagline: Smart Analysis. Simple Answers. Location: d:\mis_2\LinkScout Server: Run START_SERVER.bat or combined_server.py Extension: Load extension folder in Chrome/Edge

Enjoy your combined pro extension! πŸš€βœ¨


Created by combining the best of both worlds - MIS (Groq) + MIS_2 (Pre-trained Models)