legal contact
 

weblogs

Web Access Logs (IIS)

Traffic generated

sourcetype="whatever" website="www.xxx.com" | timechart span=15m sum(bytes_in), sum(bytes_out)

List of page visits of www.xxx.com, including least popular user-agent and HTTP methods used

sourcetype="whatever" website="www.xxx.com"  | rex field=_raw "^(?<date>.+?) (?<time>.+?) (?<www_server_name>.+?) (?<www_server_sitename>.+?) (?<www_server_ip>.+?) (?<http_method>.+?) (?<uri>.+?) (?<uri_query>.+?) (?<dst_port>.+?) (?<user_name>.+?) (?<src_ip>.+?) HTTP\/(?<http_version>.+?) (?<user_agent>.+?) (?<referrer>.+?) (?<www_domain>.+?) (?<http_status>\d+?) (?<http_win32_status>\d+?) (?<bytes_sent>\d+?) (?<bytes_recd>\d+?) (?<time_taken>.+?)" | rex field=user_agent mode=sed "s/\+/ /g" | timechart span=1h count

List the least popular User-Agents

sourcetype="whatever" website="www.xxx.com"  | rex field=_raw "^(?<date>.+?) (?<time>.+?) (?<www_server_name>.+?) (?<www_server_sitename>.+?) (?<www_server_ip>.+?) (?<http_method>.+?) (?<uri>.+?) (?<uri_query>.+?) (?<dst_port>.+?) (?<user_name>.+?) (?<src_ip>.+?) HTTP\/(?<http_version>.+?) (?<user_agent>.+?) (?<referrer>.+?) (?<www_domain>.+?) (?<http_status>\d+?) (?<http_win32_status>\d+?) (?<bytes_sent>\d+?) (?<bytes_recd>\d+?) (?<time_taken>.+?)" | rex field=user_agent mode=sed "s/\+/ /g" | search user_agent!=*bot/* user_agent!=msnbot*  user_agent!=Dalvik/* user_agent!=*spider* user_agent!=Feedfetcher-Google* |  iplocation prefix=src_ip_ src_ip | transaction clientip,user_agent maxspan=30s maxpause=5s  | rare limit=50 user_agent

List the used HTTP Methods

sourcetype="whatever" website="www.xxx.com"  | rex field=_raw "^(?<date>.+?) (?<time>.+?) (?<www_server_name>.+?) (?<www_server_sitename>.+?) (?<www_server_ip>.+?) (?<http_method>.+?) (?<uri>.+?) (?<uri_query>.+?) (?<dst_port>.+?) (?<user_name>.+?) (?<src_ip>.+?) HTTP\/(?<http_version>.+?) (?<user_agent>.+?) (?<referrer>.+?) (?<www_domain>.+?) (?<http_status>\d+?) (?<http_win32_status>\d+?) (?<bytes_sent>\d+?) (?<bytes_recd>\d+?) (?<time_taken>.+?)" | rex field=user_agent mode=sed "s/\+/ /g" | search user_agent!=*bot/* user_agent!=msnbot* |  iplocation prefix=src_ip_ src_ip | transaction clientip,user_agent maxspan=30s maxpause=5s  | top limit=50 http_method

List the friendly bots only

sourcetype="whatever" website="www.xxx.com"  | rex field=_raw "^(?<date>.+?) (?<time>.+?) (?<www_server_name>.+?) (?<www_server_sitename>.+?) (?<www_server_ip>.+?) (?<http_method>.+?) (?<uri>.+?) (?<uri_query>.+?) (?<dst_port>.+?) (?<user_name>.+?) (?<src_ip>.+?) HTTP\/(?<http_version>.+?) (?<user_agent>.+?) (?<referrer>.+?) (?<www_domain>.+?) (?<http_status>\d+?) (?<http_win32_status>\d+?) (?<bytes_sent>\d+?) (?<bytes_recd>\d+?) (?<time_taken>.+?)" | rex field=user_agent mode=sed "s/\+/ /g" | search user_agent!=*bot/* user_agent!=msnbot* user_agent!=*crawler* user_agent=*spider* user_agent!=adsbot*  |  iplocation prefix=src_ip_ src_ip | transaction clientip,user_agent maxspan=30s maxpause=5s  | top limit=50 user_agent

List all unusual bots unlegit  User-Agents

sourcetype="whatever" website="www.xxx.com"  | rex field=_raw "^(?&lt;date&gt;.+?) (?&lt;time&gt;.+?) (?&lt;www_server_name&gt;.+?) (?&lt;www_server_sitename&gt;.+?) (?&lt;www_server_ip&gt;.+?) (?&lt;http_method&gt;.+?) (?&lt;uri&gt;.+?) (?&lt;uri_query&gt;.+?) (?&lt;dst_port&gt;.+?) (?&lt;user_name&gt;.+?) (?&lt;src_ip&gt;.+?) HTTP\/(?&lt;http_version&gt;.+?) (?&lt;user_agent&gt;.+?) (?&lt;referrer&gt;.+?) (?&lt;www_domain&gt;.+?) (?&lt;http_status&gt;\d+?) (?&lt;http_win32_status&gt;\d+?) (?&lt;bytes_sent&gt;\d+?) (?&lt;bytes_recd&gt;\d+?) (?&lt;time_taken&gt;.+?)" | rex field=user_agent mode=sed "s/\+/ /g" | search user_agent!=Mozilla/*  user_agent!=Opera/* user_agent!=*CFNetwork* user_agent!=*bot/* user_agent!=msnbot* user_agent!=*crawler* user_agent!=*spider* |  iplocation prefix=src_ip_ src_ip | transaction clientip,user_agent maxspan=30s maxpause=5s  | top limit=50 user_agent</query>

Traffic profile by 24h with 15 min buckets

sourcetype="whatever" website="www.xxx.com" | timechart span=15m sum(bytes_in), sum(bytes_out)

Events over last 24h with 1h buckets

sourcetype="whatever" website="www.xxx.com"  | rex field=_raw "^(?&lt;date&gt;.+?) (?&lt;time&gt;.+?) (?&lt;www_server_name&gt;.+?) (?&lt;www_server_sitename&gt;.+?) (?&lt;www_server_ip&gt;.+?) (?&lt;http_method&gt;.+?) (?&lt;uri&gt;.+?) (?&lt;uri_query&gt;.+?) (?&lt;dst_port&gt;.+?) (?&lt;user_name&gt;.+?) (?&lt;src_ip&gt;.+?) HTTP\/(?&lt;http_version&gt;.+?) (?&lt;user_agent&gt;.+?) (?&lt;referrer&gt;.+?) (?&lt;www_domain&gt;.+?) (?&lt;http_status&gt;\d+?) (?&lt;http_win32_status&gt;\d+?) (?&lt;bytes_sent&gt;\d+?) (?&lt;bytes_recd&gt;\d+?) (?&lt;time_taken&gt;.+?)" | rex field=user_agent mode=sed "s/\+/ /g" | timechart span=1h count

List the page visits of www.xxx.com, including least popular user-agents and HTTP methods used

sourcetype="whatever" website="www.xxx.com"  | rex field=_raw "^(?&lt;date&gt;.+?) (?&lt;time&gt;.+?) (?&lt;www_server_name&gt;.+?) (?&lt;www_server_sitename&gt;.+?) (?&lt;www_server_ip&gt;.+?) (?&lt;http_method&gt;.+?) (?&lt;uri&gt;.+?) (?&lt;uri_query&gt;.+?) (?&lt;dst_port&gt;.+?) (?&lt;user_name&gt;.+?) (?&lt;src_ip&gt;.+?) HTTP\/(?&lt;http_version&gt;.+?) (?&lt;user_agent&gt;.+?) (?&lt;referrer&gt;.+?) (?&lt;www_domain&gt;.+?) (?&lt;http_status&gt;\d+?) (?&lt;http_win32_status&gt;\d+?) (?&lt;bytes_sent&gt;\d+?) (?&lt;bytes_recd&gt;\d+?) (?&lt;time_taken&gt;.+?)" | rex field=user_agent mode=sed "s/\+/ /g" | search user_agent!=*bot/* user_agent!=msnbot* user_agent!=*crawler* |  iplocation prefix=src_ip_ src_ip | transaction clientip,user_agent maxspan=30s maxpause=5s  | table  _time, src_ip, src_ip_Country, website, http_method, uri, uri_query, user_agent