SENTINAL — Signal Intelligence & Electronic Warfare Analysis System

01 — Overview

What is SENTINAL?

A full-stack Electronic Warfare Analysis System built from the ground up — from synthetic sensor data generation through machine learning classification to a tactical intelligence dashboard.

ELINT Collection

Detects, locates, and classifies adversary radar emitters using passive RF sensing across a 100x100 km tactical grid. Four geographically distributed sensors per EW system capture pulse descriptor words with realistic Gaussian noise models including freq, PRI, PW, AoA, and Doppler shift.

COMINT Analysis

Monitors adversary communication networks across VHF, UHF and S-Band frequencies. Detects frequency hopping, jamming events, and network blackouts. Online Isolation Forest provides adaptive anomaly detection that learns the baseline distribution in real time across the first 25 timesteps.

SIGINT Correlation

Cross-correlates ELINT tracks with COMINT network clusters using a composite scoring system evaluating spatial colocation, frequency band match, and co-jamming signatures. Pairs scoring 0.65+ are flagged as likely same platform, revealing radar-equipped command units.

Mission Management

Automatically detects, classifies, and tracks 10 adversary tactical mission types including STRIKE_PACKAGE, COMBAT_AIR_PATROL, and ELECTRONIC_WARFARE. Threat scores (0.0-1.0) computed from a weighted formula across entity count, speed, jamming, and heading coherence.

Common Operating Picture (COP)

A unified timestamped situational display integrating sensor data, Kalman-tracked emitter trails, COMINT network statistics, Electronic Order of Battle, SIGINT correlation pairs, jamming analysis (JAU), and a Media Intelligence pipeline with YOLOv5 video object detection and NLP text analysis — all served through 50+ REST API endpoints to a vanilla JS tactical dashboard with a dark military-grade aesthetic including scanline overlay, CRT typography, and FEBA boundary rendering.

02 — Capabilities

Full Capability Matrix

Every intelligence function, the technology powering it, and its tactical output to the operator.

Capability	Technology	Output
Radar Emitter Detection	DBSCANPDW Clustering	Emitter positions (x,y) with uncertainty bounds from sensor fusion
Radar Classification	Random Forest300 Trees	4 radar types: GS / Naval / AEW / Fighter + confidence score
Position Estimation	AoA TriangulationLeast-Squares	Geolocated emitter coordinates from multi-sensor angle bearings
Track Continuity	Kalman FilterConstant Velocity	Smoothed tracks [x,y,vx,vy] with prediction and trail history
COMINT Clustering	Agglomerative120 MHz threshold	Network groups with SNR, BER, traffic load, modulation stats
Anomaly Detection	Isolation ForestOnline Learning	Flagged anomalous transmissions vs 25-timestep learned baseline
Mission Detection	Rule-Based ClassifierMulti-Criterion	Named missions with threat scores mapping to LOW/MEDIUM/HIGH/CRITICAL
SIGINT Correlation	Spatial + Freq Scoring	Radar-COMINT platform associations flagged at score >= 0.65
Video Intelligence	YOLOv5Kalman Box Tracker	Object-tracked annotated H.264 video with persistent track IDs
Text Intelligence	NLTK NERVADER Sentiment	Named entities, frequency-ranked keywords, compound sentiment scores
Jamming Analysis (JAU)	Pre/Post-Jam Comparison	Jamming effectiveness score, SNR degradation, recovery time estimate
Electronic Order of Battle	EOB BuilderFull Aggregation	Per-emitter intelligence picture across all 300 simulation frames

03 — Architecture

5-Layer System Architecture

A pipeline architecture where sensor data flows from synthetic generation through Kafka streaming, ML intelligence processing, PostgreSQL persistence, and out to the tactical dashboard.

LAYER 01

Data Generation

Python / NumPy / Pandas

Synthetic EW sensor CSV generation. 300 timesteps at DT=1s. Realistic Gaussian noise. Scripted tactical events (jamming, anomaly, blackout).

LAYER 02

Kafka Streaming

Apache Kafka / Docker

Real-time message broker on ew1.raw + ew2.raw topics. 20MB message limit. Timestamp-grouped JSON batches. Decouples producer from consumer.

LAYER 03

Intelligence Engine

Flask / scikit-learn / NumPy

9-step ML processing pipeline per frame — DBSCAN, Random Forest, AoA triangulation, Kalman tracking, COMINT clustering, mission management.

LAYER 04

PostgreSQL

PostgreSQL / psycopg2

3-table JSONB schema: frames, system_summaries, analysis_cache. Bulk upsert with execute_values at page_size=200 for efficiency.

LAYER 05

Dashboard + Adaptor

Vanilla JS / HTML / CSS

Tactical COP dashboard (index.html) + ICIC Interface Software (adaptor.html) for operator-controlled ELT pipeline execution.

Dual EW System Configuration

EW System 1

Sensors at map corners for maximum AoA coverage. 4 emitters: 2 static, 1 moving AEW on sinusoidal racetrack orbit, 1 fighter. 2 COMINT networks (NET_A VHF ~890MHz, NET_B UHF ~1800MHz). Display colour: cyan. DBSCAN epsilon: 0.3, association: 15km.

EW System 2

Trapezoid sensor layout for asymmetric realistic coverage. 8 emitters including 3 fighter jets moving diagonally at 0.3-0.4 km/s. 3 COMINT networks (+NET_C S-Band ~3100MHz). Display colour: green. PDW signatures tuned to match EW1 classifier training distribution.

05 — Machine Learning

Intelligence Engine Deep Dive

The Flask backend runs a 9-step processing pipeline over all 300 timesteps at startup, fusing raw RF sensor data into mission-level tactical intelligence.

STEP 01

Data Loading & Feature Extraction

CSV loaded into pandas, sorted by timestamp. COMINT version detected. StandardScaler fitted on CLUSTER_FEATURES for DBSCAN normalisation across frequency, PRI, pulse width, bandwidth, scan rate, duty cycle.

STEP 02

Sensor Fusion Window

5-timestep temporal window accumulates PDWs from t-4 to t, increasing available pulse descriptors and reducing false negatives for low-PRF radars that may not trigger every timestep.

STEP 03

DBSCAN Clustering

Groups PDWs into emitter clusters using density-based clustering on normalised feature space. Points labelled -1 (noise) are discarded. Each cluster represents one radar emitter.

eps=0.3 | min_samples=3 | noise_label=-1

STEP 04

Random Forest Classification

300 decision trees classify each cluster from 9 PDW features. Returns radar_type (0-3), confidence (0-100%), and human-readable radar_name. Platform_type intentionally excluded to simulate blind real-world classification.

n_estimators=300 | max_depth=20 | accuracy ≥ 95%

STEP 05

AoA Triangulation

Least-squares triangulation from multi-sensor angle-of-arrival measurements estimates 2D emitter position. AoA residual computed as a quality metric. Requires minimum 2 sensors per emitter cluster.

nx=-sin(AoA_i), ny=cos(AoA_i) — solve Ax=b

STEP 06

COMINT Block Processing

Agglomerative clustering groups emitters by comm_frequency (120 MHz threshold). Isolation Forest trained online during first 25 timesteps to learn COMINT feature baseline. Post-training: anomalous clusters flagged at score=-1.

STEP 07

Kalman Tracking

Constant-velocity Kalman filter maintains track continuity across timesteps with nearest-neighbour gating association within 15km (EW1) or 12km (EW2). State vector: [x, y, vx, vy].

Q=0.08 | R=2.0 | trail subsampled every 3rd frame

STEP 08

Mission Management

Three-stage pipeline: COMINT-based track grouping with 45km radius merging, rule-based mission type classification from entity composition, then weighted threat score formula mapping to 4 threat levels.

score = base + 0.12*jam + 0.08*anomaly + ...

STEP 09

Frame Cache & Persistence

Each processed timestep stored in in-memory frame_cache dictionary, then bulk-upserted to PostgreSQL using execute_values (page_size=200) for efficient write throughput.

System Architecture

Data Processing Pipeline

Real-time ingestion and transformation sequence converting raw RF pulse descriptors into mission-level actionable intelligence.

          01 Kafka Stream → 
          02 DBSCAN Clustering 

          03 Random Forest → 
          04 Kalman Tracking 

          05 PostgreSQL Persistence
        

Model Evaluation

Classification Metrics

High-fidelity performance across four distinct radar emitter categories, validated against the secondary holdout dataset.

GS (Ground): 98.2%

Naval: 96.5%

AEW (Airborne): 94.1%

Fighter: 95.8%

Overall System Accuracy: ≥ 95%

Random Forest Classifier

9-Feature PDW Classification

Trained on EW1 data, generalises to EW2 — EW2 fighter PDW signatures were specifically tuned to match EW1 type-3 distribution during dataset design.

frequency | pri | pulse_width | bandwidth | scan_rate
duty_cycle | modulation_type | doppler | snr
→ radar_type in {0:GS, 1:Naval, 2:AEW, 3:Fighter}

Isolation Forest (Online)

Adaptive Anomaly Detection

Trains in real-time during the first 25 timesteps, establishing a COMINT baseline before switching to anomaly scoring mode for the remaining 275 frames.

learning_window=25 timesteps
anomaly trigger: NET_B freq drift at t=140-160
jamming trigger: SNR drop 5-12dB at t=150-170

06 — Mission Classification

10 Tactical Mission Types

The MissionManager classifies adversary group activity into named mission types based on entity composition, speed, COMINT activity, and jamming signatures.

STRIKE PACKAGE

Base Threat: 0.85

ELECTRONIC WARFARE

Base Threat: 0.78

FORCE PACKAGE

Base Threat: 0.70

COMBAT AIR PATROL

Base Threat: 0.62

AIR DEFENSE

Base Threat: 0.50

FIGHTER PATROL

Base Threat: 0.40

NAVAL OPERATION

Base Threat: 0.42

ISR SURVEILLANCE

Base Threat: 0.28

GROUND SURVEILLANCE

Base Threat: 0.22

UNKNOWN

Base Threat: 0.20

Threat Score Formula

score = base_threat(mission_type)
    + 0.12 x jamming_active
    + 0.08 x anomaly_detected
    + min(fighter_count x 0.05, 0.20)
    + min(avg_speed x 0.07, 0.10)
    + min((entity_count-1) x 0.025, 0.10)
    + heading_coherence x 0.06

Threat levels: LOW (<0.30) / MEDIUM (<0.55) / HIGH (<0.75) / CRITICAL (>=0.75)

07 — COMINT Events

Scripted Tactical Event Timeline

Three scripted events embedded in the simulation test the system's analytical robustness across the 300-timestep operational window.

Timestep

Event

Status

t = 140 - 160

Frequency Anomaly

NET_B drifts 600-1000 MHz above its normal UHF band. comm_anomaly flag set. Isolation Forest triggers anomaly detection alert.

ANOMALY

t = 150 - 170

Communications Jamming

NET_A and NET_B experience 5-12dB SNR degradation. comm_jammed=True. JAU computes jamming effectiveness and recovery time post-event.

JAMMED

t = 200 - 230

Communication Blackout

All COMINT networks go silent. comm_active=False across NET_A and NET_B. Dashboard COMINT panel shows zero active network nodes.

BLACKOUT

08 — Tech Stack

Technologies Used

A carefully selected set of technologies covering ML, streaming, backend API, database, computer vision, and NLP — no frontend frameworks used.

Backend

Flask

REST API — 50+ endpoints, CORS-first

scikit-learn

Random Forest, Isolation Forest, DBSCAN, Agglomerative

Streaming

Apache Kafka

ew1.raw + ew2.raw topics, 20MB messages

Database

PostgreSQL

JSONB schema, bulk upsert, psycopg2

Computer Vision

YOLOv5

Object detection + Kalman Box Tracker

NLP

NLTK + VADER

NER, keywords, sentiment, clustering

Data

NumPy / Pandas

Feature engineering, simulation physics

Infra

Docker

Kafka + Zookeeper + Kafka UI containers

Video

OpenCV + ffmpeg

H.264 MP4, HTTP Range seeking

Frontend

Vanilla JS

Zero frameworks — pure HTML/CSS/JS

Tracking

SciPy

Hungarian algorithm for multi-object assignment

Model I/O

joblib

Radar classifier .pkl persistence + loading

09 — Deployment

7-Step Startup Guide

From cold start to fully operational tactical intelligence dashboard with live data flowing through every panel.

Start Kafka Infrastructure

Launch Kafka broker, Zookeeper, and Kafka UI on kafka-net Docker network. Wait 20-30 seconds. UI at :8080.

docker compose up -d

Generate Datasets

Run both generators to produce EW1 (~4800 rows) and EW2 (~14400 rows) CSV files with 300 timesteps of sensor data.

python datasetgeneration_v6_advancecomint.py

Train Classifier

First time only — trains Random Forest on EW1 PDW data. Saves radar_classifier_v2.pkl. Achieves 95%+ accuracy.

python train_classifier_v2.py

Stream via Kafka

Optional — publishes both CSV datasets to their respective topics, simulating real-time sensor feed ingestion.

python kafka_producer.py

Start Backend Server

Flask server starts on :5000 in adaptor-controlled mode — RAM empty until operator triggers ELT pipeline.

python elint_server_new.py

Run ELT via Adaptor

Open adaptor.html. Set API URL. Click Run ELT — triggers pg-reprocess (30-120s) then ingest/reload sequence.

open adaptor.html → Run ELT Pipeline

Access Dashboard

Open index.html. All 12 panels are live. Navigate 300 timesteps with manual stepping or auto-play mode.

open index.html

PYTHON

3.10+

DOCKER

Latest Desktop

POSTGRESQL

13+

FFMPEG

Optional (H.264)

NODE.JS

16+ (optional)

10 — Project Files

Complete File Index

.py

datasetgeneration_v6_advancecomint.py

EW System 1 generator — seed 42, 4 emitters, corner sensors, 300 timesteps

.py

data_generation_v9_EW2.py

EW System 2 generator — seed 99, 8 emitters including 3 mobile fighter jets

.py

train_classifier_v2.py

Random Forest radar classifier training — produces radar_classifier_v2.pkl

.py

kafka_producer.py

Streams CSV data to ew1.raw + ew2.raw Kafka topics with timestamp batching

.yml

docker-compose.yml

Kafka + Zookeeper + Kafka UI on kafka-net bridge with 20MB message limit

.py

elint_server_new.py

Main Flask intelligence API — full 9-step processing pipeline, 50+ endpoints

.py

media_blueprint.py

Flask Blueprint — YOLOv5 video detection, image analysis, NLP text intelligence

.html

adaptor.html

ICIC Interface Software — standalone ELT controller and status monitoring

.html

index.html

Tactical dashboard shell — 12 intelligence panels, timeline scrubber, auto-play

.css

style.css

Frontend CSS design system — dark military aesthetic, scanline CRT overlay

.js

script.js

Frontend application — REST API client, frame rendering, playback engine

.pkl

radar_classifier_v2.pkl

Trained Random Forest model loaded by backend at startup via joblib

11 — Limitations & Roadmap

Known Limitations & Planned Enhancements

Current Limitations

Synthetic Data Only — Simplified physics models; real deployment needs calibrated antenna patterns and multi-path propagation.

Single-Node Kafka — Replication factor 1; production deployment requires multi-broker cluster for fault tolerance.

In-Memory Mission State — MissionManager state not persisted between server restarts; mission IDs reset.

JIM/JIS Stubs — CDR analysis and IMSI tracking endpoints return simulated data pending classified input.

No Real-Time Kafka Consumer — Backend reads CSV directly; true streaming requires a live consumer thread.

Single-Threaded ELT — Sequential processing; parallelisation would significantly cut the 30-120s reprocess window.

Planned Enhancements

Real Kafka Consumer — True streaming intelligence pipeline with live sensor feed ingestion via consumer thread.

Multi-Hypothesis Tracking (MHT) — Handle track ambiguity in dense emitter environments without losing continuity.

Frequency Deinterleaving — Separate interleaved pulses from co-channel emitters sharing the same frequency band.

3D Position via ToA — Time-of-arrival multilateration to add altitude estimation alongside AoA triangulation.

BERT-Based NER — Higher NER accuracy on military text over NLTK's rule-based chunking approach.

GIS Integration — Real geographic coordinates, terrain overlays, and map tile integration for COP display.

12 — Glossary

Key Terminology

SENTINAL

Signal Intelligence and Electronic Warfare Analysis System — project codename

ELINT

Electronic Intelligence — from non-communication RF emissions (radar pulses)

COMINT

Communications Intelligence — from intercepted adversary communications networks

SIGINT

Signals Intelligence — umbrella covering all electromagnetic signal interception

PDW

Pulse Descriptor Word — measured parameters characterising a single radar pulse

AoA

Angle of Arrival — bearing angle measured from sensor to transmitter

COP

Common Operating Picture — unified situational awareness display for commanders

EOB

Electronic Order of Battle — per-emitter catalogue of adversary electronic assets

JAU

Jamming Analysis Unit — analyses COMINT jamming effectiveness and recovery time

FEBA

Forward Edge of the Battle Area — tactical boundary line rendered on the COP

AEW

Airborne Early Warning — high-altitude radar aircraft providing long-range detection

ELT

Extract Load Transform — the data pipeline controlled by the ICIC adaptor

REP

Recognized Electronic Picture — comprehensive view of the electromagnetic environment

DBSCAN

Density-Based Spatial Clustering of Applications with Noise — emitter grouping algorithm

ISR

Intelligence Surveillance and Reconnaissance — passive monitoring mission type

PRI

Pulse Repetition Interval — time between consecutive radar pulses, measured in microseconds

VADER

Valence Aware Dictionary and sEntiment Reasoner — NLP sentiment analysis tool

ICIC

Interface and Control — the adaptor software that authorises ELT pipeline execution

SENTINAL Signal Intelligence

What is SENTINAL?

ELINT Collection

COMINT Analysis

SIGINT Correlation

Mission Management

Common Operating Picture (COP)

Full Capability Matrix

5-Layer System Architecture

Dual EW System Configuration

EW System 1

EW System 2

Tactical Dashboard

Intelligence Engine Deep Dive

Data Processing Pipeline

Classification Metrics

9-Feature PDW Classification

Adaptive Anomaly Detection

10 Tactical Mission Types

Threat Score Formula

Scripted Tactical Event Timeline

Technologies Used

7-Step Startup Guide

Complete File Index

Known Limitations & Planned Enhancements

Current Limitations

Planned Enhancements

Key Terminology