Telegram OSINT in python
Documentation: Telegram OSINT Script
This documentation outlines the usage of the Telegram OSINT (Open Source Intelligence) script to collect and analyze Telegram messages from groups and channels you are a member of. The script allows you to collect messages, translate them, download attachments (excluding images and videos), and analyze them using YARA rules. The results are saved in CSV files optimized for Splunk analysis.
Table of Contents
- Introduction
- Requirements
- Installation
- Configuration
- Usage
- CSV Output
- Integration with Splunk
- Customizing YARA Rules
- Troubleshooting
- Security Considerations
- License
Introduction
The Telegram OSINT script is a tool designed to collect messages from Telegram groups and channels. It provides the following features:
- Message Collection: Download messages from Telegram groups/channels.
- Translation: Translate messages into English.
- File Downloads: Download attachments (excluding images and videos).
- YARA Rule Matching: Analyze messages using YARA rules to identify specific content.
- CSV Output: Save results in CSV files optimized for further analysis with Splunk.
Requirements
- Python 3.6+
- Telethon: Python library for interacting with the Telegram API.
- googletrans: Library for translation services.
- YARA-Python: Library for using YARA rules.
- Splunk (optional): For analyzing the generated CSV files.
Installation
-
Install Python (if not already installed):
sudo apt-get update sudo apt-get install python3 python3-pip
-
Install required Python packages:
pip3 install telethon googletrans==4.0.0-rc1 yara-python
Configuration
Telegram API Keys
-
Obtain Telegram API keys:
- Visit my.telegram.org and sign in with your phone number.
- Click "API development tools."
- Create a new application to get your
api_id
andapi_hash
.
-
Create the
telegram.api
file:Create a file named
telegram.api
in the same directory as the script and add the following:api_id = YOUR_API_ID api_hash = 'YOUR_API_HASH' phone_number = '+YOUR_PHONE_NUMBER'
Replace
YOUR_API_ID
,YOUR_API_HASH
, and+YOUR_PHONE_NUMBER
with your actual data.
YARA Rules
-
Create a directory for YARA rules:
mkdir /data/yara
-
Add YARA rules:
-
Create
.yar
files in the/data/yara
directory. -
Example rule (
financial_sector_germany_extended_en.yar
):rule FinancialSectorAttack_Germany { meta: description = "Indicators of potential attacks or malicious activities targeting the financial sector in Germany" author = "Your Name" date = "YYYY-MM-DD" strings: $bank1 = "Deutsche Bank" $attack1 = "hack the bank" // ... other strings ... condition: any of them }
-
Usage
Running the Script
Save the final script as telegram-osint.py
and run it with Python 3:
python3 telegram-osint.py
The script will:
- Collect messages from the groups/channels you are a member of.
- Translate the messages and analyze them using YARA rules.
- Save the results in CSV files located in
telegram_data/messages/
.
Automating with systemd
To automate the script (e.g., running every 60 minutes), use a systemd timer.
-
Create the service file
/etc/systemd/system/telegram-daemon.service
:[Unit] Description=Telegram Daemon for fetching messages After=network.target [Service] Type=simple ExecStart=/usr/bin/python3 /path/to/telegram-osint.py WorkingDirectory=/path/to Restart=on-failure RestartSec=60 [Install] WantedBy=multi-user.target
Note: Replace
/path/to
with the actual path to the script. -
Create the timer file
/etc/systemd/system/telegram-daemon.timer
:[Unit] Description=Run telegram daemon every 60 minutes [Timer] OnBootSec=10min OnUnitActiveSec=60min Unit=telegram-daemon.service [Install] WantedBy=timers.target
-
Enable and start the timer:
sudo systemctl enable telegram-daemon.timer sudo systemctl start telegram-daemon.timer
CSV Output
The CSV files are saved in the directory telegram_data/messages/
and include the following fields:
- time: Timestamp of the message in the format
YYYY-MM-DDTHH:MM:SS
. - message_id: Unique ID of the message.
- sender_id: Telegram ID of the sender.
- username: Username of the sender.
- first_name: First name of the sender.
- last_name: Last name of the sender.
- phone: Phone number of the sender (if available).
- is_bot: Whether the sender is a bot.
- is_verified: Whether the sender is verified.
- is_restricted: Whether the sender is restricted.
- is_scam: Whether the sender is flagged as a scammer.
- message: Original message text.
- translated_message: Translated message in English.
- media_file: Path to the downloaded attachment (if any).
- yara_match: Names of the YARA rules that matched the message (if any).
Integration with Splunk
Creating a Splunk Sourcetype
To import the CSV files into Splunk:
-
Create a new sourcetype in Splunk with the following settings:
-
Sourcetype Name:
telegram_messages
-
Settings:
[telegram_messages] DATETIME_CONFIG = CURRENT INDEXED_EXTRACTIONS = csv FIELD_NAMES = time,message_id,sender_id,username,first_name,last_name,phone,is_bot,is_verified,is_restricted,is_scam,message,translated_message,media_file,yara_match FIELD_TYPES = time:timestamp, message_id:int, sender_id:int, username:string, first_name:string, last_name:string, phone:string, is_bot:boolean, is_verified:boolean, is_restricted:boolean, is_scam:boolean, message:text, translated_message:text, media_file:string, yara_match:string
-
-
Import CSV files into Splunk:
- Use the created sourcetype
telegram_messages
to import the CSV files. - Splunk will automatically recognize the fields for efficient data analysis.
- Use the created sourcetype
Customizing YARA Rules
- Adding new rules: Add more
.yar
files in the/data/yara
directory. - Updating rules: Edit existing YARA rules to search for new patterns or threats.
- Example: To extend a rule for financial sector attacks, you can add more keywords related to known attacks.
Troubleshooting
- YARA errors: Ensure all YARA rules are syntactically correct. Use
yara.compile()
to properly load the rule files. - Telegram API issues: Verify your
api_id
,api_hash
, andphone_number
in thetelegram.api
file. - Dependencies: Ensure that all required Python packages are installed.
Security Considerations
- Data Privacy: Ensure that you do not collect or share personal or sensitive data without proper authorization.
- Telegram Policies: Be aware of Telegram's terms of service and privacy policies.
- Responsible Use: Use this script for lawful and ethical purposes only.
License
This script and accompanying documentation are released under the MIT License.
Note: This documentation was created with the intent to provide clear instructions for using and customizing the Telegram OSINT script. Should you encounter any issues or need further assistance, feel free to reach out to the author or the community for support.