automatic Telegram monitroing for all the nerdy stuff

Abstract

The inclined reader may think whatever he wants of Telegram and its many groups. However, it is undisputed that since the Ukrine crisis at the latest, it's a nice source to find valuable data and information.

Whether the quality is justified according to use as an OSINT source can be technically checked and correlated. But let's start with a practical example from the Ukraine crisis just mentioned.

During the crisis, Telegram, with its various pro-Russian and pro-Ukraine groups, was repeatedly the focus of observations to identify any cyber attack targets in its own environment. The various actions (mostly DDoS and database/information leaks) carried out by targets in the non-German environment were also published here.

original post anouncing an attack.

This made Telegram a quite valuable source for the various status reports on the respective threat situation in our IT landscape. At least something you had to look at to report a complete picture of the situation.

Only the amount of time involved in monitoring the various channels had become too intensive given the low level of threat. Especially since the posts were mostly in Russian or other languages that German analysts could hardly understand.

My dream of (partially) automating the evaluation was quickly born.

There should be a solution that recognizes a user's subscribed groups and automatically examines them for "treats" without having to experience a lot of manual work from the analyst in this phase.

Subject to an ongoing data protection review and assessment by specialists from FI-TS, I have initiated a "feasibility study" for the time being. However, as soon as the "OK" is received from the departments, we begin to professionalize the service and take it over into operation.

First step: Entering the subscribed groups of a (my) user

Here the "telegram-cli" was a valuable alternative to the Telegram client. With a simple command, a list of subscribed groups can be displayed here and added to a local "Target Groups" file. This has m.U. the advantage that by simply subscribing to a group in the regular Telegram client by the analyst, a future analysis can take place.

Second step: downloading the messages of the "Target Groups"

Using "telegram-messages-dump" proved very helpful to accomplish this task. In particular, the fact that there is the option of only backing up the last (new) messages and, above all, the supported JSON format are of elementary importance for further processing with Splunk for the IOC search.

Third step: translate into English

However, all posts and text were still written in the respective language, so that an analysis was not possible without the appropriate knowledge. - But Google's deepL with its translation AI can help here.

The clever "translate-shell" automatically recognizes the respective source language and translates it from all supported languages into English. How ingenious, now it is possible to import the translated texts in the JSON file into Splunk in order to search there in the already existing IOC engine for corresponding occurrences of interest.

Fourth step: search for indicators within the existing IOC processes

Since only the actual text in the JSON is translated, the meta information is retained and it is possible in the Splunk evaluation to identify the corresponding Telegram message so that the analyst can look at the information again in the original Telegram post.

Like others, the corresponding findings are included in the daily report and give the analyst direct feedback on findings that affect him. - All without having to read a single post or analyze/translate. (smile)

Fifth step: Checking leak data from the different groups

The different groups in Telegram m.U. so-called "leaks". In order to prove the validity and value of such data leaks, smaller or larger data samples are often made available. Entire databases are also published free of charge to improve the reputation of the individual groups.

These databases usually contain email addresses/user IDs and passwords from various websites that are of interest. If a user ID and password is found there with a correspondingly clear reference to us or our customers, it is generally at least a policy violation (no company addresses for private activities), but at least a vague suspicion that the same (similar) password structure for the Internal company purpose is used. That is, the owner of that user ID should be contacted and asked to change their passwords.

Long story short effect. The leak files published in Telegram are downloaded using the "telegram_media_downloader".

Since these files m.U. The tool offers the possibility to view only relevant file types and on top of that ensures that files are only downloaded once. - Even if the program is called up periodically.

Currently, the files still have to be checked manually, since the data structure in particular cannot (yet) be automatically recognized for a successful import into Splunk.

In view of the quality of this leak data, however, automation will soon be established, as I can save a lot of time here.

The results of the automation