Sentinel — Multimodal AI Event Processing Prototype

Sentinel is an educational prototype that demonstrates how a multimodal AI system can ingest, analyze and summarize heterogeneous data streams in real time.
The project is inspired by conceptual ideas seen in fictional films such as Person of Interest, but this implementation is strictly a technical experiment focused on concurrency, pipelines and AI preprocessing.

Sentinel does not perform real surveillance and does not operate on private or sensitive data.
All data sources used during development were simulated or generated locally.

⚡ Overview

Sentinel continuously receives "feeds" from multiple heterogeneous sources, such as:

• video footages (camera feeds)
• audio files
• short messages
• GPS-like position samples
• simulated news articles
• etc. etc.

Each source is handled by a dedicated worker thread.
Threads ingest data, perform AI-based preprocessing (object detection, transcription, simple NLP, etc.) and forward structured results to synchronized queues protected by mutexes and semaphores.

After filtering, each feed is converted into a feed summary, a short textual description representing a meaningful event associated with a specific user (e.g., "User A was seen arguing in a crowd").

These summaries are vectorized and stored for each user.

Although the "Feed Summaries" generation (from feeds) and full "Core Heuristic" learning module are not completed, the intended design is:

1. News articles describing negative events are analyzed (sentiment & danger features).
2. The system identifies the people involved in the described event
3. Their historical feed summaries form the input vector.
4. The danger sentiment vector of the article forms the target vector.
5. The central neural network uses these vectors (input and target) to gradually learn correlations between behavioral patterns and high-risk events.

Over time, the system would infer which users exhibit patterns similar to those seen in past dangerous events — effectively ranking users by a predicted "risk score".

🧱 Architecture Summary

• Multithreading: one thread per feeds source
• Synchronization: mutexes + semaphores for queue handling
• Pipelines: ingest → preprocess → summarize → vectorize
• Multimodal AI preprocessing:
  • object/person detection (video)
  • speech transcription (audio)
  • sentiment/NLP (text)
• Planned: neural network to associate behavior vectors with risk labels extracted from news events
• Execution model: continuous real-time flow

Even as a partial prototype, Sentinel demonstrates the interaction between concurrency, AI preprocessing and event-driven architectures.

🛠 Installation

Sentinel uses environment.yml to define required Python packages.

1. Create the environment

conda env create -f environment.yml

1. Activate the environment

conda activate ai_general

3. Run

python ./sentinel.py

🔒 Ethical Disclaimer

This project is strictly educational and intended to explore:

• concurrency models

• AI preprocessing pipelines

• multimodal data handling

Sentinel must not be used for real-world surveillance, monitoring of individuals, or any activity that violates privacy, ethics, or applicable laws. The author rejects and discourages any misuse of this code outside of academic, experimental and personal learning contexts.

📄 License

This project is released under the GPLv3 license. See the LICENSE file for details.
Copyright (C) 2025 Davide Quirillo