legal contact rss

automatic Telegram monitroing for all the nerdy stuff

All the data in Splunk for comprehensive IOC-matchings.


The inclined reader may think whatever he wants of Telegram and its many groups. However, it is undisputed that since the Ukrine crisis at the latest, it's a nice source to find valuable data and information.

Whether the quality is justified according to use as an OSINT source can be technically checked and correlated. But let's start with a practical example from the Ukraine crisis just mentioned.

During the crisis, Telegram, with its various pro-Russian and pro-Ukraine groups, was repeatedly the focus of observations to identify any cyber attack targets in its own environment. The various actions (mostly DDoS and database/information leaks) carried out by targets in the non-German environment were also published here.

original post anouncing an attack.

This made Telegram a quite valuable source for the various status reports on the respective threat situation in our IT landscape. At least something you had to look at to report a complete picture of the situation.

Only the amount of time involved in monitoring the various channels had become too intensive given the low level of threat. Especially since the posts were mostly in Russian or other languages that German analysts could hardly understand.

My dream of (partially) automating the evaluation was quickly born.

There should be a solution that recognizes a user's subscribed groups and automatically examines them for "treats" without having to experience a lot of manual work from the analyst in this phase.

Subject to an ongoing data protection review and assessment by specialists from FI-TS, I have initiated a "feasibility study" for the time being. However, as soon as the "OK" is received from the departments, we begin to professionalize the service and take it over into operation.

First step: Entering the subscribed groups of a (my) user

Here the "telegram-cli" was a valuable alternative to the Telegram client. With a simple command, a list of subscribed groups can be displayed here and added to a local "Target Groups" file. This has m.U. the advantage that by simply subscribing to a group in the regular Telegram client by the analyst, a future analysis can take place.

Second step: downloading the messages of the "Target Groups"

Using "telegram-messages-dump" proved very helpful to accomplish this task. In particular, the fact that there is the option of only backing up the last (new) messages and, above all, the supported JSON format are of elementary importance for further processing with Splunk for the IOC search.

Third step: translate into English

However, all posts and text were still written in the respective language, so that an analysis was not possible without the appropriate knowledge. - But Google's deepL with its translation AI can help here.

The clever "translate-shell" automatically recognizes the respective source language and translates it from all supported languages into English. How ingenious, now it is possible to import the translated texts in the JSON file into Splunk in order to search there in the already existing IOC engine for corresponding occurrences of interest.

Fourth step: search for indicators within the existing IOC processes

Since only the actual text in the JSON is translated, the meta information is retained and it is possible in the Splunk evaluation to identify the corresponding Telegram message so that the analyst can look at the information again in the original Telegram post.

Like others, the corresponding findings are included in the daily report and give the analyst direct feedback on findings that affect him. - All without having to read a single post or analyze/translate. (smile)

Fifth step: Checking leak data from the different groups

The different groups in Telegram m.U. so-called "leaks". In order to prove the validity and value of such data leaks, smaller or larger data samples are often made available. Entire databases are also published free of charge to improve the reputation of the individual groups.

These databases usually contain email addresses/user IDs and passwords from various websites that are of interest. If a user ID and password is found there with a correspondingly clear reference to us or our customers, it is generally at least a policy violation (no company addresses for private activities), but at least a vague suspicion that the same (similar) password structure for the Internal company purpose is used. That is, the owner of that user ID should be contacted and asked to change their passwords.

Long story short effect. The leak files published in Telegram are downloaded using the "telegram_media_downloader".

Since these files m.U. The tool offers the possibility to view only relevant file types and on top of that ensures that files are only downloaded once. - Even if the program is called up periodically.

Currently, the files still have to be checked manually, since the data structure in particular cannot (yet) be automatically recognized for a successful import into Splunk.

In view of the quality of this leak data, however, automation will soon be established, as I can save a lot of time here.

The results of the automation

Providing a cool dashboard with some statistical data in Splunk.
Since the data ultimately ends up in Splunk using various configurations, corresponding evaluations are possible. - The examples listed here are only initial results intended to show the possibilities and do not correspond to the ultimate functionality of the application.

Total of the imported leak data processed as a file attachment in the messages. The usernames and associated passwords found are reported here immediately. Note the logarithmic style of the graph here. On one day (07/24/22) more than 10 million leaked accounts with corresponding passwords were imported and could be automatically checked for matches with indicators from the FI-TS environment.

A so-called IOC watch list in which there are various keywords for data worthy of protection. Any occurrences of such indicators in the Telegram messages are displayed here and also sent as a security alert for further processing. Here, for example, various DDoS attacks on German facilities as part of the Ukraine crisis were announced days before they were to be carried out.

The analysis of the file attachments contained in some Telegram messages is also automated and provides a daily report with the indicators via alert or mail. Unfortunately, I can only show exemplary data here, but rest assured that the lists often contain more than 100 entries per day .
Just to show, that the DDoS attacks against some german facilities were anounced via Telegram.