AI-Powered Automated Cybersecurity Threat Detection Platform

Watcher is a Django & React JS platform designed to discover and monitor emerging cybersecurity threats with AI-powered threat intelligence analysis. It can be deployed on webservers or quickly run via Docker.

Watcher Capabilities

Watcher empowers your security operations with comprehensive threat detection and monitoring:

• AI-Driven Threat Intelligence - Transform raw threat data into actionable intelligence with automated weekly digests of top-5 trending cybersecurity topics, real-time breaking news alerts when threats emerge, on-demand summaries for any security keyword including related CVE and threat actor details.

• CVE & Ransomware Intelligence - Continuously fetch, correlate, and surface external threat data: CVEs from cve.circl.lu, ransomware victims and groups from ransomware.live and ransomlook.io. Define keyword-based Watch Rules to get alerted when specific threats match your organisation's context.

• Emerging Threat Detection - Monitor cybersecurity trends via RSS feeds from CERT-FR (www.cert.ssi.gouv.fr), CERT-EU (www.cert.europa.eu), US-CERT (www.us-cert.gov), Australian Cyber Security Centre (www.cyber.gov.au), and more. Track new vulnerabilities, malware campaigns, and threat advisories as they appear.

• Legitimate Domain Management - Centralized approved domains with expiry, repurchase status, registrar info, and contacts. Easily convert monitored malicious domains into legitimate ones. Automated UDRP case tracking.

• Information Leak Monitoring - Detect sensitive data exposure across the webs including Pastebin, StackOverflow, GitHub, GitLab, Bitbucket, APKMirror, npm registries, and other platforms. Catch leaked credentials, API keys, and confidential information early.

• Malicious Domain Surveillance - Monitor malicious domains for changes in IP addresses, mail/MX records, and web content. Use TLSH fuzzy hashing to detect modifications. Automatic RDAP/WHOIS checks with registrar and expiry alerts.

• Suspicious Domain Detection - Identify potentially malicious domains targeting your organisation via:

• Domain Generation Algorithm Detection using dnstwist to find typosquatting, homograph attacks, and similar domain variants
• Certificate Transparency Monitoring via certstream to catch newly registered suspicious domains in real-time

Additional Features

Extend Watcher's capabilities with powerful integrations and management tools:

• TheHive Full Synchronization - Integration with TheHive featuring automated alert creation, smart case management, IOC enrichment, and ready-to-use Cortex Analyzers & Responders. Detailed configuration are provided in the documentation here.
• MISP Integration - Seamlessly export Indicators of Compromise (IOCs) to MISP with smart UUID tracking, automatic object creation, and manual attribute updates for collaborative threat intelligence sharing
• SSO / OpenID Connect - Federated login via any OIDC provider (Keycloak, Azure AD, etc.) with PKCE and Knox token issuance. Configurable per-instance via .env
• Bulk Add & Bulk Actions - Add many items at once with the One item per line toggle (with a no-write Preview and a per-item created / skipped / error summary), or tick rows in any table to edit, delete or export them (to Legitimate Domains or MISP) in one go
• Flexible Authentication - Support for LDAP, local, and SSO/OIDC authentication systems
• Connectors Dashboard - A superuser-only /connectors page to view, edit, and test every external integration (SMTP, Slack, Citadel, TheHive, MISP, CyberWatch feeds, and more) from one place, with encrypted credential storage and per-connector health checks.
• Smart Notifications - Receive email, Slack, or Citadel alerts for critical findings and threshold violations across all modules including CyberWatch and UDRP decisions
• Interactive API Documentation - Auto-generated Swagger UI at /api/docs/ and OpenAPI 3 schema at /api/schema/ powered by drf-spectacular
• Ticketing System Integration - Automatically feed your ticketing system with security findings
• Comprehensive Admin Interface - Manage all aspects of Watcher through Django's powerful admin panel
• Advanced Access Control - Granular user permissions and group management for team collaboration
• Modern UI Experience - A modern interface with customizable themes, resizable dashboard panels, advanced filtering with saved filter sets, and persistent user preferences

Involved dependencies

Watcher leverages open source tools and libraries: - Hugging Face Transformers - AI/ML framework powering threat intelligence summarization and entity extraction - google/flan-t5-base - Text-to-text generation model for AI-powered threat summaries - dslim/bert-base-NER - Named Entity Recognition for automatic IOC extraction - certstream - Certificate Transparency monitoring - dnstwist - Domain name permutation engine - SearxNG - Privacy-respecting metasearch engine - PyMISP - MISP threat intelligence platform integration - TLSH - Fuzzy hashing for content similarity detection - mozilla-django-oidc - OpenID Connect authentication backend - drf-spectacular - OpenAPI 3 schema generation and Swagger UI - deck.gl - WebGL-powered geospatial visualization (WorldMap) - MUI (Material UI) - React component library for KPI cards and charts - shadow-useragent - User-Agent rotation library - NLTK - Natural Language Toolkit for text processing

App Preview

All the screenshots below use fictional data (a made-up company, "Altair Dynamics").

Overview

A tour of the platform: the AI weekly report and breaking news, the threat dashboard, legitimate and suspicious domains, DNS threats, data leaks and Cyber Watch.

Threat Detection

AI-Powered Weekly Summary & Breaking News

Suspicious domain names detection

Legitimate Domain List

Add several domains in one go with the One item per line toggle: defanged links are restored, a Preview shows the result before anything is saved, and the summary lists every domain as created, skipped or in error.

Data Leak Detection

Suspicious domain names monitoring

Select several domains and change legitimacy, takedown, legal and blocking flags together.

Bulk actions and exports

False positives can be exported to Legitimate Domains in bulk: the exported sites disappear from Website Monitoring and appear in the legitimate list, with no page refresh.

Cyber Watch

Watch rules, monitored keywords and RSS sources, with the CVE and ransomware matches they produce.

History of every change

Every create, update and delete is recorded with its author, so you can see who changed what and when.

Customisable dashboards

Show, hide, move and resize panels: the layout is saved in your profile.

Theme Previews

Watcher offers multiple visual themes to match your preferences and working environment.

Admin Interface

Django provides a ready-to-use user interface for administrative activities. We all know how an admin interface is important for a web project: Users management, user group management, Watcher configuration, usage logs...

Installation

# 1. Clone the repo
git clone https://github.com/thalesgroup-cert/watcher.git
cd watcher/deployment

# 2. Initialize environment, configs & directory structure
make init

# 3. Start the stack
make up

# 4. On first run: run database migrations + create superuser
make migrate
make superuser
make populate-db

# 5. Open the web UI
#    http://localhost:9002  (or your configured domain/port)

Get Watcher up and running in just 10 minutes using Docker. Detailed instructions available in our Installation Guide

Platform Architecture

flowchart LR
    subgraph SRC["External sources"]
        RSS["RSS / CERT advisories"]
        CVE["CVE & ransomware feeds"]
        CT["Certificate Transparency"]
        WEB["Paste sites & code registries"]
        RSS ~~~ CVE ~~~ CT ~~~ WEB
    end

    subgraph HOST["Host server (Docker)"]
        TRAEFIK["Traefik<br/>TLS reverse proxy"]
        subgraph WATCHER["Watcher (Django)"]
            UI["React UI / Django admin"]
            API["Django REST API"]
            JOBS["Scheduled jobs (APScheduler)<br/>Threats · CyberWatch · DNS Threats<br/>Data Leak · Website Monitoring"]
            CONN["Connectors"]
        end
        CERTSTREAM["CertStream"]
        SEARX["SearXNG"]
        DB[("MySQL")]
    end

    subgraph OUT["Outputs"]
        HIVE["TheHive"]
        MISP["MISP"]
        CHAT["Slack / Citadel"]
        MAIL["Email (SMTP)"]
    end

    ANALYST(["Analyst"]) --> TRAEFIK --> UI
    UI <--> API
    API <--> DB
    JOBS <--> DB
    RSS & CVE --> JOBS
    CT --> CERTSTREAM --> JOBS
    WEB --> SEARX --> JOBS
    JOBS --> CONN
    API --> CONN
    CONN --> HIVE & MISP & CHAT & MAIL

Watcher's modular architecture ensures scalability, reliability, and easy integration with your existing security stack.

Contributing

We welcome contributions from the security community!

To report bugs, request features, or submit code, please read our full Contributing Guide.

Pastebin Compliance

In order to use Watcher pastebin API feature, you need to subscribe to a pastebin pro account and whitelist Watcher public IP (see https://pastebin.com/doc_scraping_api).