OSINT
Open Source INTeligence refers to the gathering and analysis of information from open sources to create useful intelligence. In a classic penetration test it will be our first step but in CTF we may not need this if we only have a machine to pwn. This skill is very important because it allows us to get a better understanding of the network we are facing.
Passive Information Gathering
Our first step will be Passive Information Gathering. Indeed, we don’t want to be spotted by our client from the start so first we are going to act like any other user or service (clicking links, navigating on websites, check for DNS entries…).
WHOIS
WHOIS is a query and response protocol that is popularly used for accessing databases that list the registered users or assignees of Internet resources, including domain names, blocks of IP addresses, or autonomous systems, but is also used for a larger range of other information.
We are going to use this command on the github website. This command will allows us to get mails, phone numbers, name servers and much more.
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
└─$ whois github.com
Domain Name: GITHUB.COM
Registry Domain ID: 1264983250_DOMAIN_COM-VRSN
Registrar WHOIS Server: whois.markmonitor.com
Registrar URL: http://www.markmonitor.com
Updated Date: 2020-09-08T09:18:27Z
Creation Date: 2007-10-09T18:20:50Z
Registry Expiry Date: 2022-10-09T18:20:50Z
Registrar: MarkMonitor Inc.
Registrar IANA ID: 292
Registrar Abuse Contact Email: abusecomplaints@markmonitor.com
Registrar Abuse Contact Phone: +1.2086851750
Domain Status: clientDeleteProhibited https://icann.org/epp#clientDeleteProhibited
Domain Status: clientTransferProhibited https://icann.org/epp#clientTransferProhibited
Domain Status: clientUpdateProhibited https://icann.org/epp#clientUpdateProhibited
Name Server: DNS1.P08.NSONE.NET
Name Server: DNS2.P08.NSONE.NET
Name Server: DNS3.P08.NSONE.NET
Name Server: DNS4.P08.NSONE.NET
Name Server: NS-1283.AWSDNS-32.ORG
Name Server: NS-1707.AWSDNS-21.CO.UK
Name Server: NS-421.AWSDNS-52.COM
Name Server: NS-520.AWSDNS-01.NET
DNSSEC: unsigned
<SNIP>
Note that this command is also available on
windows CMD
DNS
“You talked about DNS but what is this ?”. Domain Name System is a service (running on port 53) that allows us to look for github.com instead of its IP. For us, human, it’s easier to remember words that IPs that are succession of numbers. Where us are going to read the FQDN github.com, DNS are going to start with the com and then github.
For more information about DNS you can check this video
NSLOOKUP and DIG
The nslookup command allows us to query name server of a domain. We can get IPs of the domain we are looking for :
1
2
3
4
5
6
7
8
└─$ nslookup github.com
Server: 192.168.111.2
Address: 192.168.111.2#53
Non-authoritative answer:
Name: github.com
Address: 140.82.121.3
The dig command is very similar to nslookup. We can specify a name server with the syntax @name_server:
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
└─$ dig github.com
; <<>> DiG 9.18.0-2-Debian <<>> github.com
;; global options: +cmd
;; Got answer:
;; ->>HEADER<<- opcode: QUERY, status: NOERROR, id: 41519
;; flags: qr rd ra; QUERY: 1, ANSWER: 1, AUTHORITY: 0, ADDITIONAL: 1
;; OPT PSEUDOSECTION:
; EDNS: version: 0, flags:; MBZ: 0x0005, udp: 1232
; COOKIE: 77571a6e301260fd01000000630f655833f8151aea163960 (good)
;; QUESTION SECTION:
;github.com. IN A
;; ANSWER SECTION:
github.com. 5 IN A 140.82.121.3
;; Query time: 28 msec
;; SERVER: 192.168.111.2#53(192.168.111.2) (UDP)
;; WHEN: Wed Aug 31 09:43:28 EDT 2022
;; MSG SIZE rcvd: 83
With both nslookup and dig we can query DNS records (TXT, A, AAAA, MX…). With nslookup you just have to add the -query= flag followed by the DNS record you want to query (ex: nslookup -query=TXT TARGET. We can do the same with dig by adding the DNS record after the dig command like : dig AAAA TARGET.
Google Dorking
| Symbol / Syntax | Function |
|---|---|
"search phrase" | Find results with exact search phrase |
OSINT filetype:pdf | Find files of type PDF related to a certain term (here OSINT). |
salary site:example.com | Limit search results to a specific site. |
pentest -site:example.com | Exclude a specific site from results |
walkthrough intitle:example | Find pages with a specific term in the page title (here walkthrough). |
challenge inurl:example | Find pages with a specific term in the page URL (here challenge). |
Others
You can search for useful information on the victim websites, look for employees on their social media accounts (Facebook, Twitter, LinkedIn…). Many other automated tools exist like the following one :
| Name | Usefulness |
|---|---|
| Virus Total | DNS information gathering |
| Censys | Certificates information gathering |
| CRT | Certificates information gathering |
| osintframework | Lots of useful tools |
| The Harvester | Gather emails, names, subdomains, IP addresses, and URLs |
| The Way Back Machine | Access to older version of a website and possibly removed content |
| Shodan | Look for IoT arround the world |
| WhatsMyName | Username finder |
| Sherlock | Username finder |
| Hunter.io | Mail finder |
| Wigle | Retrieve information about a WIFI |
| Maltego | Information gathering about a company or a person (can become active info gathering) |
| Recon-ng | Web enumeration gatherer |
Active Information Gathering
Now that we have some informations about our target we may want to get more but this time by interacting with their systems.
Web Information Gathering
With WhatWeb you can get information about the web server (including CMS if one is used), Wappalyzer will display this result directly on the navigator and WafW00f will gives us informations about the firewall in place.
Subdomain Enumeration
A secondary DNS server receives information from the primary DNS server and changes it through a zone transfer. A domain’s DNS servers are organized using the master-slave model, with the slave getting updated DNS data from the master DNS. Zone transfers from secondary (slave) DNS servers should be permitted by the master DNS server, although this might be misconfigured.
You can find Zone Transfer example here. You can use the dig or the nslookup command for this. Here is an example with the nslookup command because on the previously mantioned website we can see it with the dig command:
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
nslookup -type=any -query=AXFR zonetransfer.me nsztm1.digi.ninja
<SNIP>
Server: nsztm1.digi.ninja
Address: 81.4.108.41#53
printsection()
zonetransfer.me
origin = nsztm1.digi.ninja
mail addr = robin.digi.ninja
serial = 2019100801
refresh = 172800
retry = 900
expire = 1209600
minimum = 3600
zonetransfer.me hinfo = "Casio fx-700G" "Windows XP"
zonetransfer.me text = "google-site-verification=tyP28J7JAUHA9fw2sHXMgcCC0I6XBmmoVi04VlMewxA"
zonetransfer.me mail exchanger = 0 ASPMX.L.GOOGLE.COM.
zonetransfer.me mail exchanger = 10 ALT1.ASPMX.L.GOOGLE.COM.
zonetransfer.me mail exchanger = 10 ALT2.ASPMX.L.GOOGLE.COM.
zonetransfer.me mail exchanger = 20 ASPMX2.GOOGLEMAIL.COM.
zonetransfer.me mail exchanger = 20 ASPMX3.GOOGLEMAIL.COM.
zonetransfer.me mail exchanger = 20 ASPMX4.GOOGLEMAIL.COM.
zonetransfer.me mail exchanger = 20 ASPMX5.GOOGLEMAIL.COM.
<SNIP>
You can also list subdomains with the gobuster tool like this :
1
gobuster dns -q -r NS_SERVER_NAME -d DOMAIN -w WORDLIST
Virtual Hosts Discovery
Multiple domain names can be hosted on a single server using the virtual hosting technique (each name is handled separately) . By doing this, one server may share resources like memory and processing time without having that each service it offers have the same host name.
We can use the Ffuf tool to do so. First we need to know the size of response that doesn’t exist. You can do so by using BurpSuite, cURL or just launching ffuf and stop it after a few seconds just to get the size of the vhost that doesn’t exist and put it after the -fs flag.
1
ffuf -w WORDLIST -u http://SITE -H "HOST: FUZZ.DOMAIN.com" -fs NUMBER
GEOINT / IMINT
This skills may not be usefull when working in a company but can be utile in CTFs or for national agencies. GEOINT is an intelligence field that uses geospatial information and data to define, evaluate, and visually represent physical characteristics and geographically based actions on Earth. Whereas IMINT is a discipline of intelligence collection that analyzes imagery to find information with intelligence value.
In those disciplines, you need to pay attention to every detail (languages of signs, stores, licence plates…). A very useful tool is Yandex. It’s the Russian version of Google but their AI to analyse images is much more powerful.