AIL framework: Analysis Information Leak framework

by ddos · December 9, 2024

AIL framework

AIL framework – Framework for Analysis of Information Leaks

AIL is a modular framework to analyse potential information leaks from unstructured data sources like pastes from Pastebin or similar services or unstructured data streams. AIL framework is flexible and can be extended to support other functionalities to mine or process sensitive information (e.g. data leak prevention).

Trending charts

Features

Modular architecture to handle streams of unstructured or structured information
Default support for external ZMQ feeds, such as provided by CIRCL or other providers
Multiple Importers and feeds support

Each module can process and reprocess the information already analyzed by AIL
Detecting and extracting URLs including their geographical location (e.g. IP address location)
Extracting and validating potential leaks of credit card numbers, credentials, …

Extracting and validating leaked email addresses, including DNS MX validation
Module for extracting Tor .onion addresses for further analysis
Keep tracks of credentials duplicates (and diffing between each duplicate found)

Extracting and validating potential hostnames (e.g. to feed Passive DNS systems)
A full-text indexer module to index unstructured information
Terms, Set of terms, Regex, typo squatting and YARA tracking and occurrence

YARA Retro Hunt
Many more modules for extracting phone numbers, credentials, and more
Alerting to MISP to share found leaks within a threat intelligence platform using MISP standard

Detecting and decoding encoded file (Base64, hex encoded or your own decoding scheme) and storing files
Detecting Amazon AWS and Google API keys
Detecting Bitcoin address and Bitcoin private keys

Detecting private keys, certificate, keys (including SSH, OpenVPN)
Detecting IBAN bank accounts
Tagging system with MISP Galaxy and MISP Taxonomies tags

UI submission
Create events on MISP and cases on The Hive
Automatic export on detection with MISP (events) and The Hive (alerts) on selected tags

Extracted and decoded files can be searched by date range, type of file (mime-type) and encoding discovered
Correlations engine and Graph to visualize relationships between decoded files (hashes), PGP UIDs, domains, username, and cryptocurrencies addresses
Websites, Forums and Tor Hidden-Services hidden services crawler to crawl and parse output

Domain availability monitoring to detect up and down of websites and hidden services
Browsed hidden services are automatically captured and integrated into the analyzed output, including a blurring screenshot interface (to avoid “burning the eyes” of security analysts with sensitive content)
Tor hidden services is part of the standard framework, all the AIL modules are available to the crawled hidden services