<h1>Advanced Reconnaissance with Recon-ng (Part III)</h1>

Now is a good time to filter all those unwanted contacts from unwanted domains. This can be done using SQL statements on the SQLite database.

Filtering and cleaning data

• Let’s see how many contacts we have from the united.com domain

db query SELECT email FROM contacts WHERE email LIKE “%@united.com”

• But we also a lot of contacts from other domains

db query SELECT email FROM contacts WHERE email NOT LIKE “%@united.com”

• And some others don’t have an e-mail address

 db query SELECT email FROM contacts WHERE email IS NULL

• 
  

• Let’s delete all contacts from unwanted domains

db query DELETE FROM contacts WHERE email NOT LIKE “%@united.com”

Let’s try to create email addresses for the contacts only having first and last names

• This can be done using another specific module named mangle

modules load recon/contacts-contacts/mangle

• Set the proper module options

The result is not perfect but it is better than before because now all contacts have an e-mail address.

Let’s now give some attention to the credentials table.

• Let’s see how many credentials we have from unwanted domains

db query SELECT username FROM credentials WHERE username NOT LIKE “%@united.com”

• Let’s delete all this useless information

db query DELETE FROM credentials WHERE username NOT LIKE “%@united.com”

Let’s keep cleaning the information found.

• Are there duplicates in the contacts table ?

db query SELECT first_name,last_name FROM contacts WHERE rowid NOT IN (SELECT MIN(rowid) FROM contacts GROUP BY email)

• Let’s delete all duplicates

db query DELETE FROM contacts WHERE rowid NOT IN (SELECT MIN(rowid) FROM contacts GROUP BY email)

Let’s repeat the previous procedure for the credentials table

• List duplicates

db query SELECT * FROM credentials WHERE rowid NOT IN (SELECT MIN(rowid) FROM credentials GROUP BY username)

• Delete all duplicates

db query DELETE FROM credentials WHERE rowid NOT IN (SELECT MIN(rowid) FROM credentials GROUP BY username)

What about the profiles table?

• List duplicates

db query SELECT * FROM profiles WHERE rowid NOT IN (SELECT MIN(rowid) FROM profiles GROUP BY username)

• Do we have repeated urls?

db query SELECT * from profiles WHERE rowid NOT IN(SELECT MIN(rowid) from profiles GROUP BY url)

• But we have many repeated usernames pointing to different profiles. Those are all useless.

db query SELECT * from profiles WHERE rowid NOT IN(SELECT MIN(rowid) from profiles GROUP BY username)

• Let’s delete all the repeated usernames

db query DELETE from profiles WHERE rowid NOT IN(SELECT MIN(rowid) from profiles GROUP BY username)

• Another analysis of the profiles shows that some are just referring to the country or language of the profile

db query SELECT * from profiles WHERE username LIKE "__" OR username LIKE "__-__"

• Remove these profiles

db query DELETE from profiles WHERE username LIKE "__" OR username LIKE "__-__"

It might be a good idea to make another database snapshot.

Finding additional data

Now we have a set of “clean” data but many records in the credentials table only have the hashes.

• Search for credentials with no passwords.

db query SELECT * from credentials WHERE password ISNULL

There is a module that tries to find known hashes.

• Load the hashes module and run it

modules load recon/credentials-credentials/hashes_org

run

• If you want to focus on a specific group of hashes, set the module options

options set SOURCE query SELECT DISTINCT hash FROM credentials WHERE hash LIKE “__ <number of characters in the hash>_” AND password ISNULL AND leak LIKE “linkedin.com”

The result will be the discovery of almost 200 additional passwords.

Initially, we used the bing_linkedin_cache module to find contacts. That is a “companies to contacts” module.

Now we can query Linkedin directly using the Bing API via the bing_linkedin_contacts module. And that is a “profiles to contacts” modules.

NOTE: Don’t forget to make a database snapshot!

• Load and run the module

modules load recon/profiles-contacts/bing_linkedin_contacts

This module will generate a lot of duplicate data because the new information will be inserted into new records. However, you will get a lot of information about the current positions of the company’s employees. The module might crash but that is because you will probably exceed the number of allowed queries with a free Bing API…

If you are skilled in SQL you can try to merge those records. Anyway, we already have a lot of profiles but they are limited to Linkedin. We can try to get some more from a wide variety of social media and popular websites.

It will be a very slow process because each individual profile will be tested against dozens of websites.

• Load and run the profiler module

modules load recon/profiles-profiles/profiler

Once you are happy with the results, you can export them in multiple formats using one of the reporting modules.

Conclusion: With only a few free APIs, and using nothing but open-source data, it is possible to gather a huge amount of information.