The Recon-ng Framework

Link: http://resources.infosecinstitute.com/the-recon-ng-framework-automated-information-gathering/

The Metasploit Framework Project and the Social Engineer Toolkit (SET) are two great and known frameworks used by penetration testers and security researchers for automation wherein the former is used for automated exploitation of known vulnerabilities while the latter is used for penetration testing by hacking a user with the use of social engineering. These are very helpful tools indeed! For security enthusiasts out there, I have good news for you because there is another tool that has been unleashed just recently with a new purpose! Let me present to you the new ‘Recon-ng Framework’!

Recon-ng is an open-source framework coded in python by Tim Tomes a.k.a LaNMaSteR53. Its interface is modeled after the look of the Metasploit Framework but it is not for exploitation or for spawning a meterpreter session or a shell, it is for web-based reconnaissance and information gathering. It comes with modules to support your web reconnaissance adventure and information gathering just like Metasploit’s auxiliary and exploit modules. The modules pre-loaded for this framework are categorized into Auxiliary, Contacts, Hosts, Output, and Pwnedlist module types.

Auxiliary

The auxiliary modules include:

auxiliary_elmah – a’elmah.axd’ log web page checker
auxiliary_googli – performs a reverse hash lookup with the use of Goog.li hash database
auxiliary_mangle – applies a mangle pattern to all of the contacts stored in the database, creating email addresses or usernames for each harvested contact
auxiliary_noisette – performs a reverse hash lookup with the use of Noisette.ch hash database
auxiliary_pwnedlist – uses PwnedList.com to check if an email account is compromised.
auxiliary_resolve – resolves IP addresses to hosts
auxiliary_server_status – a server-status web page checker

Contacts

The contacts module type contains modules for harvesting and discovering contact information of a certain company through keywords. The modules include:

contacts_jigsaw – information harvesting with the use of Jigsaw.com
contacts_linkedin_auth – information harvesting through LinkedIn.com using an authenticated connections network

Output

The modules in the Output category extracts the results and stores them in a CSV file by using the output_csvfilemodule or in an HTML file with the use of the output_htmlfile module.

Hosts

The modules included in this category allow host discovery wherein you can check additional domains of a certain website. Here are the modules for the Host category:

hosts_baidu – Baidu Hostname Enumerator
hosts_bing – Bing Hostname Enumerator
hosts_brute_force – DNS Hostname Brute Forcer
hosts_google – Google Hostname Enumerator
hosts_netcraft – Netcraft Hostname Enumerator
hosts_shodan – Shodan Hostname Enumerator
hosts_yahoo – Yahoo Hostname Enumerator

Pwnedlist

The modules associated with the Pwnedlist category uses the Pwnedlist.com API (Application Programming Interface) to get full credentials and details of compromised user accounts. Hence, giving users an easy access to stolen information and credentials of pawned accounts. Here are the modules for this category:

pwnedlist_account_creds – PwnedList Account Credentials Fetcher
pwnedlist_api_usage – PwnedList API Usage Statistics Fetcher
pwnedlist_domain_creds – PwnedList Pwned Domain Credentials Fetcher
pwnedlist_domain_ispwned – PwnedList Pwned Domain Statistics Fetcher
pwnedlist_leak_lookup – PwnedList Leak Details Fetcher

Installing and Running Recon-ng

Recon-ng is not yet included in BackTrack 5 r3, BackBox Linux 3.0, Nodezero Linux, or other Linux penetration testing distributions out there but it can be manually installed using git, just open your terminal emulator and type:

git clone https://bitbucket.org/LaNMaSteR53/recon-ng.git

To launch the Recon-ng script type:

cd recon-ng

./recon-ng.py

Basic Usage of Recon-ng

Now that we have the framework up and running we type help or ? on its interface to see the available commands. Below are the results of the help menu:

back – Exits the current prompt level

banner – Displays the banner

exit – Exits current prompt level

help – Displays the menu which lists all the commands

info – Displays the module information

load – Loads the selected module

modules – Lists all available modules

options – Lists the options

query – Queries the database

reload – Reloads all the modules

schema – Displays the database schema

search – Searches available modules

set – Sets global options

shell – Executes shell commands

use – Loads the selected module (has the same functionality with the load command)

Suppose we want to use the auxiliary_server_status module under the Auxiliary type of module, we can just type:

load auxiliary_server_status or use auxiliary_server_status

Now let’s check the description and information about the auxiliary_server_status module by typing on the prompt info auxiliary_server_status or just info, because we have already loaded the said module. Below is the description of the module:

Name:

Apache Server-Status Page Scanner

Author:

Tim Tomes (@LaNMaSteR53)

Description:

Checks all of the hosts stored in the database for a ‘server-status’ page.

Options:

Name Current Value Req Description

——- ——————- —— —————————

source database yes source of module input

verbose True yes verbose output

Comments:

Source options: database, <hostname>, <path/to/infile>

http://blog.sucuri.net/2012/10/popular-sites-with-apache-server-status-enabled.html

http://httpd.apache.org/docs/2.2/mod/mod_status.html

Google dork: intitle:”Apache Status” inurl:”server-status”

Based on the description above, the Apache Server-Status Page Scanner module (auxiliary_server_status) checks if a certain website has a server-status page which allows administrators to check if the server of their website is doing well. The page shows the Server Version, CPU Usage, Active Connections, Child Server number – generation, some OS process

ID’s, and other details which are related to the Apache Server. A security researcher named Daniel Cid said, “probably not a big deal by itself (well, if you don’t have an unprotected admin panel), but that can help attackers easily find more information about these environments and use them for more complex attacks.” I agree! And this is a good thing for information gathering indeed.

“For server admins, please disable server-status or restrict it to only a set of IP addresses that really need to use it. This link explains how to do so: http://httpd.apache.org/docs/2.2/mod/mod_status.html”, he added.

Daniel Cid is also responsible for disclosing that popular websites like php.net, metacafe.com, apache.org, cisco.com, etc… have Apache server-status enabled.

What’s good about this module is that it also includes a Google dork!

To specify the configuration that needs to be set for the usage of this module we type options:

Name Current Value Req Description

——- ——————- —— —————————

source database yes source of module input

verbose True yes verbose output

Now let’s try to check if Apache.org has a server-status page that is up and enabled by setting the source to apache.org, we do this by typing set source apache.org.

Now let’s run the module by typing run in order to see the results just like the output below:

[*] http://apache.org/server-status/ => 200. Possible server status page found!

[*] https://apache.org/server-status/ => 200. Possible server status page found!

[*] 2 Server Status pages found

To exit the module, just type exit.

Let’s fire up another module called the Jigsaw Contact Enumerator (contacts_jigsaw), which harvests contact information from Jigsaw.com. As what I have heard from my sources, Jigsaw.com is a website where you can access 30 million job profiles, contact information, email addresses and other details of a person plus the website pays $1 USD for every contact information you add. LOL!

More details about the module:

recon-ng > info contacts_jigsaw

Name:

Jigsaw Contact Enumerator

Author:

Tim Tomes (@LaNMaSteR53)

Description:

Harvests contacts from Jigsaw.com. This module updates the ‘contacts’ table of the database with the results.

Options:

Name Current Value Req Description

——– —————– —– ————————————————-

company yes target company name

keywords no additional keywords to identify company

verbose True yes verbose output

Now let’s set the website of Google as our target company name:

recon-ng > use contacts_jigsaw

recon-ng [contacts_jigsaw] > set company google.com

company => google.com

recon-ng [contacts_jigsaw] > run

[*] Gathering Company IDs…

[*] Query: http://www.jigsaw.com/FreeTextSearchCompany.xhtml?opCode=search&freeText=google.com+

[*] 215043 Google, Inc. (5,896 contacts)

[*] Unique Company Match Found: 215043

[*] Gathering Contact IDs for Company ’215043′…

[*] Query: http://www.jigsaw.com/SearchContact.xhtml?rpage=1&opCode=showCompDir&companyId=215043

[*] Query: http://www.jigsaw.com/SearchContact.xhtml?rpage=2&opCode=showCompDir&companyId=215043

[*] Query: http://www.jigsaw.com/SearchContact.xhtml?rpage=3&opCode=showCompDir&companyId=215043

[*] Query: http://www.jigsaw.com/SearchContact.xhtml?rpage=4&opCode=showCompDir&companyId=215043

[*] Query: http://www.jigsaw.com/SearchContact.xhtml?rpage=5&opCode=showCompDir&companyId=215043

[*] Query: http://www.jigsaw.com/SearchContact.xhtml?rpage=6&opCode=showCompDir&companyId=215043

It would take long to harvest all the contact information of Google, so I decided to just skip the ongoing queries made by the module by hitting Ctrl+c on the keyboard. Take note that this will still gather and output the contacts that were scanned by the module. Let’s take a look on some of the results harvested by the module!

[*] Gathering Contacts…

[*] Thom Quick – Systems Engineering Manager
[*] Anne Pearce – Section Manager
[*] Troy Lyon – Documentation Manager
[*] Varma Chanderraju – Principal Staff Engineer
[*] Jennifer Wood – Manager, Sales
[*] Robert Fitton – Manager Sales
[*] Anthony Williams – Sales Manager
[*] Elizabeth Powell – Manager Consumer Behavior
[*] Alper Turken – Account Manager Networks
[*] Helen Shih – Engineer
[*] Kevin Hsu – Engineer
[*] Eric Lee – Engineer
[*] Johnson Wei – Engineer
[*] Morris Chang – Engineer
[*] Krista Meyer – Supv
[*] Lisa Zhao – Systems Administrator
[*] Art Psyhogoes – Sales Support Representative
[*] Jaya Pandey – Operations Manager-Contact Center-South West Asia
[*] Amy Haukeness – Technical Solutions
[*] Shannon Doll – Senior AE
[*] Jennifer Pham – Finance-A
[*] Michelle Calabrese – Engineering Design
[*] Sirish Kumar – South Asia Motorola Credit Corporation
[*] Dan Naylor – EIT in Shaumberg
[*] Jay White – Account Executive
[*] Leslie Patent – Executive Assistant, Creative Lab
[*] Timothy Langham – Senior SW Engineer
[*] Ramy Ayoub – Software Manager
[*] Peter Marqui – Software Engineer
[*] Giri Badanahatt – principle Staff Engineer
[*] Elizabeth Altman – Vice President, Director
[*] Ying Li – Engineering Department
[*] Abdon Badillo – Software Tech
[*] Pat Oregan – Senior Program Manager
[*] Dale Zalewski – Principal Product Manager
[*] Jan Wei – Program Manager
[*] Andrej Koperdan – Manager Corporate Development and Strategic Transa
[*] Robert Stein – Systems Engineer
[*] Stalin Subbiah – Engineer Software Staff Principal
[*] Robert Ryan – Manager Publications
[*] Ginny Machado – Engineer
[*] Mary Adnes-Ice – Inventory Disposition
[*] Eve Bills – Product Marketing
[*] David Harris – Director CDX
[*] Robert Bero – Engineer
[*] Michael O’Connor – Senior Channel Manager
[*] Anthony Braskich – Senior Research Engineer
[*] Anthony Schooler – Engineer
[*] Bob Logalbo – PRINCIPLE Staff Engineer
[*] Bob Uskali – Engineer
[*] Brian Bauerschmidt – PRINCIPLE Staff Engineer
[*] Chris Morrison – PRINCIPLE Staff Engineer
[*] Jay Gillette – Senior RF Engineer
[*] Jeff Jalowiec – PRINCIPLE Staff Engineer
[*] Joe Sepeczi – Engineering-BLACK BELT
[*] John May – Senior Presales Engineer
[*] Juan Martinez – Electrical Engineer
[*] Madjid Nakhjiri – Principal Staff Engineer
[*] Matt Defano – Engineer
[*] Matthew McIntyre – Project Manager
[*] Nitin Mangalvedhe – Principal Staff Engineer

Oh, Jigsaw! I’m not sure if it’s all updated or accurate but at least we have some leads :).

To generate a report from the contact information that were harvested by the Jigsaw Contact Enumerator module, you can load the output_csvfile module
which creates a CSV report or the output_htmlfile module if you want to create an HTML report. In my case I used the output_htmlfile module:

recon-ng > load output_htmlfile

recon-ng [output_htmlfile] > info

Name:

HTML Report Generator

Author:

Tim Tomes (@LaNMaSteR53)

Description:

Creates a HTML report.

Options:

Name Current Value Req Description

——– ———————- — ——————————————

filename ./data/results.html yes path and filename for report output

sanitize True yes mask sensitive data in the report

recon-ng [output_htmlfile] > set filename /home/shipcode/results.html

filename => /home/shipcode/results.html

recon-ng [output_htmlfile] > run

[*] Report generated at ‘/home/shipcode/results.html’.

Below is the screenshot of the ouput of the HTML file that was generated by the Recon-ng Framework.

Conclusion

In the field of ethical hacking and penetration testing, reconnaissance is very essential because we can have leads on the target and possibly find a loophole or vulnerabilities that have been overlooked. With the use of the Recon-ng Framework, our manual way of conducting reconnaissance like company website searches, Whois lookups, DNS Enumeration, Nslookup, contact information gathering, host discovery, etc… are somehow made easier and simpler because of its interface and pre-loaded modules.

Thanks LaNMaSteR53 for your contribution to open source technology, OSINT, Information Security, and the Reconnaissance Methodology.

References:

http://pentestlab.wordpress.com/2013/01/31/discover-contacts-and-domains-with-recon-ng/

http://pauldotcom.com/2013/01/the-recon-ng-framework—offic.html

http://blog.sucuri.net/2012/10/popular-sites-with-apache-server-status-enabled.html

http://techcrunch.com/2006/03/23/jigsaw-is-a-really-really-bad-idea/