Archive | October, 2008

Morris Worm To Turn 20 – How Far Things Have Come

Your website & network are Hackable


Robert Tappan Morris is a character of Internet lore, anyone who has studied Computer Science, Software Engineering or Computer Security will have heard of this guy.

He’s pretty much the fellow that made the Internet famous (for all the wrong reasons) and the first creator of a bit of self-replicating network based malware (now known as a worm) that did some serious damage. The worm is called the Morris Worm after it’s creator and this Sunday will be it’s 20 year anniversary.

The Internet will mark an infamous anniversary on Sunday, when the Morris worm turns 20. Considered the first major attack on the ‘Net, the Morris worm served as a wake-up call to the Internet engineering community about the risk of software bugs, and it set the stage for network security to become a valid area of research and development.

“It was a really big deal,” says Eric Allman, a computer programmer who in 1981 authored sendmail, open source Internet e-mail software, while he was a student at the University of California at Berkeley. Today, Allman serves as chief science officer at Sendmail, a company that sells commercial-grade versions of the software.

“The biggest implication of the Morris worm was that the Internet was very small … and it was considered a friendly place, a clubhouse,” Allman says. “This [attack] made it clear that there were some people in that clubhouse who didn’t have the best interests of the world in mind … This made it clear we had to think about security.”

It was when the Internet first got into the mainstream media and was known to the general public, albeit it was for the wrong reason. But still I doubt anyone expected back then how dependant we would eventually become on the Internet.

Now if something like this went on and 10% of the Internet was down and seriously effected by a worm which caused denial of service….the whole World would be in turmoil.

It’d be chaos seriously!

The Morris worm was written by Cornell University student Robert Tappan Morris, who was later convicted of computer fraud for the incident. Today, Morris is a respected associate professor of computer science at MIT.

Launched around 6 p.m. on Nov. 2, 1988, the Morris worm disabled approximately 10% of all Internet-connected systems, which were estimated at more than 60,000 machines.

The Morris worm was a self-replicating program that exploited known weaknesses in common utilities including sendmail, which is e-mail routing software, and Finger, a tool that showed which users were logged on to the network.

The Morris worm was able to break into Sun 3 systems and Digital VAX computers running BSD Unix. The fast-spreading worm kept copying itself and infecting computers multiple times, causing many systems to fail.

It’s an interesting story to read if you are familiar with it and probably even more interesting if you’re not, so do check out the Wiki pages on the history of the situation.

There have been very few worms that have caused such widespread infection and failures. Blaster was quite memory and on the virus front I remember CIH was terrible.

Source: Network World


Posted in: General Hacking, Malware

Tags: , , , , , , , ,

Posted in: General Hacking, Malware | Add a Comment
Recent in General Hacking:
- BADLOCK – Are ‘Branded’ Exploits Going Too Far?
- Dradis – Reporting Platform For IT Security Professionals
- Kid Gets Arrested For Building A Clock – World Goes NUTS

Related Posts:

Most Read in General Hacking:
- 10 Best Security Live CD Distros (Pen-Test, Forensics & Recovery) - 1,170,877 views
- Hack Tools/Exploits - 628,831 views
- Password Cracking with Rainbowcrack and Rainbow Tables - 435,488 views

Malwarebytes Anti-Exploit Premium | 1 Year 1 PC for $24.95


Sam Spade – Network Investigation Tool for Windows

Your website & network are Hackable


Sam Spade is one of the oldest network security tools around in terms of a neat package containing a lot of stuff you need, it’s one of the first things I used when I got into information security and I was on a crusade against spammers and scammers.

It has all kinds of useful tools in a neat graphical interface, a lot of them are available on the command line in Windows – but they aren’t so easy to use. It’s extremely useful for tracking spam or ‘UCE’ as it’s known (Unsolicited Commercial E-mail).

Some of the features included are:

  • Ping
  • NSlookup
  • Whois
  • IP block search
  • Dig
  • Traceroute
  • Finger
  • SMTP VRFY
  • Web browser keep-alive
  • DNS zone transfer
  • SMTP relay check
  • Usenet cancel check
  • Website download
  • Website search
  • Email header analysis
  • Email blacklist
  • Query Abuse address

Some other cool stuff it does is:

  • Each tool displays it’s output in it’s own window, and everything is multi-threaded so you don’t need to wait for one query to complete before starting the next one
  • Some functions are threaded still further to allow lazy reverse DNS lookups (never do a traceroute -n again)
  • The output from each query is hotlinked, so you can right click on an email address, IP address, hostname or internic tag to run another query on it
  • Appending the results of a query to the log window is a single button function
  • There’s a lot of online help, in both WinHelp and HTMLHelp formats. This includes tutorials, background information and links to online resources as well as the program manual itself

You can download Sam Spade here:

Sam Spade v1.14

Or read more here.


Posted in: General Hacking

Tags: , , , , , ,

Posted in: General Hacking | Add a Comment
Recent in General Hacking:
- BADLOCK – Are ‘Branded’ Exploits Going Too Far?
- Dradis – Reporting Platform For IT Security Professionals
- Kid Gets Arrested For Building A Clock – World Goes NUTS

Related Posts:

Most Read in General Hacking:
- 10 Best Security Live CD Distros (Pen-Test, Forensics & Recovery) - 1,170,877 views
- Hack Tools/Exploits - 628,831 views
- Password Cracking with Rainbowcrack and Rainbow Tables - 435,488 views

Malwarebytes Anti-Exploit Premium | 1 Year 1 PC for $24.95


Google Hacking Back in The News – Google Takes Action

Find your website's Achilles' Heel


Google hacking was the big thing back in 2004, I actually did a talk on it in Hack in the Box 2004, it’s resurfaced again as a serious threat with Google noticing more queries relating to things like social security numbers.

The Google Hacking Database has been active for years now and there are hundreds of queries that can bring up juicy information. Goolag was also released this year which gives a much easier, automated way of Google Hacking for specific domains or info.

Search engines such as Google are increasingly being used by hackers against Web applications that hold sensitive data, according to a security expert.

Even with rising awareness about data security, it takes all of a few seconds to pluck Social Security numbers from Web sites using targeted search terms, said Amichai Shulman, founder and CTO for database- and application-security company Imperva.

The fact that Social Security numbers are even on the Web is a human error; the information should never be published in the first place. But hackers are using Google in more sophisticated ways to automate attacks against Web sites, Shulman said.

Shulman said Imperva recently discovered a way to execute a SQL injection attack that comes from an IP address that belongs to Google.

It seems like it’s becoming big business on both sides, finding information and vulnerable sites and by gaming Google into dropping pages from the index (Blackhat SEO).

Even with the throttling it’ll still continue, people will find smarter ways to make the queries so it’s not blocked and they’ll build rate limiting into their tools so they don’t get dropped. The bad guys have plenty of patience, trust me on that.

Manipulating Google is particularly useful since it offers anonymity for a hacker plus an automated attack engine, Shulman said.

Tools such as Goolag and Gooscan can execute broad searches across the Web for specific vulnerabilities and return lists of Web sites that have those problems.

“This is no more a script kiddy game — this is a business,” Shulman said. “This is a very powerful hacking capability.”

Another attack method is so-called Google worms, which use the search engine to find specific vulnerabilities. With the inclusion of additional code, the vulnerability can be exploited, Shulman said.

“In 2004, this was science fiction,” Shulman said. “In 2008, this is a painful reality.”

Google and other search engines are taking steps to stop the abuse. For example, Google has stopped certain kinds of searches that could yield a trove of Social Security numbers in a single swoop. It also puts limits on the number of search requests sent per minute, which can slow down mass searches for vulnerable Web sites.

As they said, this is not some script kiddy stuff, with the amount of queries going on and the complexity this is some serious business!

Any pen-test or vulnerability assessment should have an information gathering stage and it’s here you should be using Google Hacking techniques and tools to uncover anything on the domain or company infrastructure that shouldn’t be there.

Just be warned that this kind of stuff is on the up, so brief your clients of the dangers and make sure this step is included in the audit.

Source: Network World


Posted in: Database Hacking, Exploits/Vulnerabilities, Privacy, Web Hacking

Tags: , , , , , , , ,

Posted in: Database Hacking, Exploits/Vulnerabilities, Privacy, Web Hacking | Add a Comment
Recent in Database Hacking:
- Onapsis Bizploit v1.50 – SAP Penetration Testing Framework
- OAT – Oracle Auditing Tools For Database Security
- ODAT (Oracle Database Attacking Tool) – Test Oracle Database Security

Related Posts:

Most Read in Database Hacking:
- Pangolin – Automatic SQL Injection Tool - 76,924 views
- bsqlbf 1.1 – Blind SQL Injection Tool - 54,484 views
- SQLBrute – SQL Injection Brute Force Tool - 41,270 views

Malwarebytes Anti-Exploit Premium | 1 Year 1 PC for $24.95


sqlmap 0.6.1 released – Automatic SQL Injection Tool

Your website & network are Hackable


sqlmap is an automatic SQL injection tool developed in Python. Its goal is to detect and take advantage of SQL injection vulnerabilities on web applications. Once it detects one or more SQL injections on the target host, the user can choose among a variety of options to perform an extensive back-end database management system fingerprint, retrieve DBMS session user and database, enumerate users, password hashes, privileges, databases, dump entire or user’s specific DBMS tables/columns, run his own SQL SELECT statement, read specific files on the file system and much more.

Features

  • Full support for MySQL, Oracle, PostgreSQL and Microsoft SQL Server back-end database management systems. Besides these four database management systems, sqlmap can also identify Microsoft Access, DB2, Informix, Sybase and Interbase.
  • Extensive back-end database management system fingerprint based upon inband error messages, banner parsing, functions output comparison and specific features such as MySQL comment injection. It is also possible to force the back-end database management system name if you already know it.
  • Full support for two SQL injection techniques: blind SQL injection and inband SQL injection.

Changes

Some of the new features include:

  • Added a Metasploit Framework 3 auxiliary module to run sqlmap;
  • Implemented possibility to test for and inject also on LIKE statements;
  • Implemented –start and –stop options to set the first and the last table entry to dump;
  • Added non-interactive/batch-mode (–batch) option to make it easy to wrap sqlmap in Metasploit and any other tool.

Complete list of changes at ChangeLog.

You can also grab the User Manual here.

You can download sqlmap 0.6.1 here:

Source – sqlmap-0.6.1.tar.gz

Windows – sqlmap-0.6.1_exe.zip

Or read more here.


Posted in: Database Hacking, Hacking Tools, Web Hacking

Tags: , , , , , , , ,

Posted in: Database Hacking, Hacking Tools, Web Hacking | Add a Comment
Recent in Database Hacking:
- Onapsis Bizploit v1.50 – SAP Penetration Testing Framework
- OAT – Oracle Auditing Tools For Database Security
- ODAT (Oracle Database Attacking Tool) – Test Oracle Database Security

Related Posts:

Most Read in Database Hacking:
- Pangolin – Automatic SQL Injection Tool - 76,924 views
- bsqlbf 1.1 – Blind SQL Injection Tool - 54,484 views
- SQLBrute – SQL Injection Brute Force Tool - 41,270 views

Malwarebytes Anti-Exploit Premium | 1 Year 1 PC for $24.95


Microsoft Rushes Out Critical RPC Bug Fix

Find your website's Achilles' Heel


Now this doesn’t happen all that often, it must be really serious! An Out-of-Band patch from Microsoft (since it’s famous ‘Patch Tuesday‘ it only releases patches on the second Tuesday of each month) has been released for a new RPC flaw.

I’d imagine it’s similar to the RPC flaw that spawned such disasters as Blaster and Sasser in 2003/4.

Microsoft Security Bulletin MS08-067 – Critical

Microsoft has released an emergency security update for a broad swath of its users that patches a critical security hole that is already being exploited in the wild.

The vulnerability – which has been subjected to “limited, targeted attacks” – could allow miscreants to create wormable exploits that remotely execute malicious code on vulnerable machines, Microsoft said. No interaction is required from the end user. It was the first patch released outside Microsoft’s regular update cycle in 18 months.

“This is a remote code execution vulnerability,” Microsoft’s out-of-band advisory warned. “An attacker who successfully exploited this vulnerability could take complete control of an affected system remotely.”

There is an active piece of malware in the wild using this, F-secure has already detected it and has a signature for Trojan-Spy:W32/Gimmiv.A.

This may have been running around in the wild for some time, perhaps in the underground community. There are always true remote exploits that are unknown to the mass community used by certain higher level groups.

This is the sixth time Microsoft has issued and out-of-band security update since October 2004 when it implemented its policy of releasing patches on the second Tuesday of each month, a company spokesman said. The last time an unscheduled patch update was issued was in April 2007 when it moved to fix a critical bug in the ANI animated cursor feature of Windows.

Thursday’s bulletin also marked the second time Microsoft has offered additional vulnerability details to security providers in advance. About an hour before the patch was released publicly, members of the Microsoft Active Protections Program (MAPP) received a briefing that allowed them to create signatures that detect exploits in anti-virus software and intrusion prevention systems.

Microsoft also offered a stunning amount of detail about the vulnerability to regular Joes here.

It’s only the 6th time this has happened since October 2004 (around 4 years) so you can see that it’s serious and you better install it across any networks you administer.

The update will require a reboot (as usual..).

Source: The Register


Posted in: Exploits/Vulnerabilities, Malware, Windows Hacking

Tags: , , , , , , , , , , , ,

Posted in: Exploits/Vulnerabilities, Malware, Windows Hacking | Add a Comment
Recent in Exploits/Vulnerabilities:
- Shadow Brokers NSA Hack Leaks 0-day Vulnerabilities
- Pompem – Exploit & Vulnerability Finder
- Bug Bounties Reaching $500,000 For iOS Exploits

Related Posts:

Most Read in Exploits/Vulnerabilities:
- Learn to use Metasploit – Tutorials, Docs & Videos - 235,432 views
- AJAX: Is your application secure enough? - 120,202 views
- eEye Launches 0-Day Exploit Tracker - 85,641 views

Malwarebytes Anti-Exploit Premium | 1 Year 1 PC for $24.95


XSS-Proxy – Cross Site Scripting Attack Tool

Find your website's Achilles' Heel


XSS-Proxy is an advanced Cross-Site-Scripting (XSS) attack tool. The documents, tools and other content on this site assume you have a basic understanding of XSS issues and existing exploitation methods. If you are not famliar with XSS, then I recommend you check out the primer links/docs below to get a better of idea of what XSS is and how to detect it, fix it, and exploit it.

Cross Site Scripting (XSS)
CERT info on XSS
CGISecurity’s Cross Site Scripting FAQ
Gunter Ollmann’s XSS paper
PeterW’s Cross Site Request Forgery (CSRF) Concept
SecureNet’s Session Riding paper

Some Common Misconceptions about XSS

  • A user has to click a link to be impacted by XSS.” No – if you visit a page that has your browser will run it regardless of you clicking a link. I carefully crafted this example so it would not be run by your browser, but I could have put real script tags/commands here and made you run then transparently.
  • XSS only matters with bulliten boards, blogs, and other sites where an attacker can upload script content.” That is one way the attack can happen, but an attacker can also leverage sites that allow HTML/SCRIPT tags to be reflected back to the same user (like a search form that repeats what it was told to look for in the response). These flaws are commonly combined with public site redirects or emails to attack a second site.
  • Don’t XSS attacks just create popup windows, alerts and other pesky things?” No – They are commonly used to reveal your cookies or form based login info to attackers. After havesting this info, the attacker uses it to log into the same site as you.
  • I understand XSS, but I don’t think it’s a huge issue“. I think you’ll change your mind once you understand this advanced attack. Read the advanced stuff below and play with XSS-Proxy to see how evil XSS really can be.

You can download XSS-Proxy here:

XSS-Proxy_0_0_12-book.pl

Or read more here.


Posted in: Hacking Tools, Web Hacking

Tags: , , , , , , , , ,

Posted in: Hacking Tools, Web Hacking | Add a Comment
Recent in Hacking Tools:
- PowerOPS – PowerShell Runspace Portable Post Exploitation Tool
- Shadow Brokers NSA Hack Leaks 0-day Vulnerabilities
- UFONet – Open Redirect DDoS Tool

Related Posts:

Most Read in Hacking Tools:
- Top 15 Security/Hacking Tools & Utilities - 1,987,183 views
- Brutus Password Cracker – Download brutus-aet2.zip AET2 - 1,456,961 views
- wwwhack 1.9 – Download wwwhack19.zip Web Hacking Tool - 684,170 views

Malwarebytes Anti-Exploit Premium | 1 Year 1 PC for $24.95


Swiss Researchers Sniff Password from Wired Keyboard

Your website & network are Hackable


Now this is an interesting twist on an oldschool method of hacking, the monitoring of electromagnetic radiation.

You’d think it’d be easier to sniff the traffic from a wireless keyboard, but generally it’s not as they tend to be encrypted. Where as the electromagnetic radiation given off by a wired keyboard is not shielded or protected it any way.

All you need to do is have the equipment and the know-how to decipher it.

Swiss researchers have demonstrated a variety of ways to eavesdrop on the sensitive messages computer users type by monitoring their wired keyboards. At least 11 models using a wide range of connection types are vulnerable.

The researchers from the Security and Cryptography Laboratory at Ecole Polytechnique Federale de Lausanne are able to capture keystrokes by monitoring the electromagnetic radiation of PS/2, universal serial bus, or laptop keyboards. They’ve outline four separate attack methods, some that work at a distance of as much as 65 feet from the target.

In one video demonstration, researchers Martin Vuagnoux and Sylvain Pasini sniff out the the keystrokes typed into a standard keyboard using a large antenna that’s about 20 to 30 feet away in an adjacent room.

It appears to work on both the older PS/2 keyboards and new USB keyboards and even laptop keyboard from a distance of up to 65 feet! That’s easily far enough to jack the data from a carpark, adjacent office or nearby hotel room.

I’d imagine the equipment required is quite bulky though.

“We conclude that wired computer keyboards sold in the stores generate compromising emanations (mainly because of the cost pressures in the design),” they write here. “Hence they are not safe to transmit sensitive information.”

No doubt, electromagnetic eavesdropping dates back to the mid 1980s, if not earlier. But Vuagnoux says many of today’s keyboards have been adapted to prevent those attacks from working. The research shows that even these keyboards are vulnerable to electromagnetic sniffing.

The video demonstrations show a computer that reads input from antennas that monitor a specified frequency. In both cases, the computer was able to determine the keystrokes typed on keyboards connected to a laptop and power supply and LCD monitors were disconnected to prevent potential power transmissions or wireless communications. Vuagnous said in an email that the attacks would still work even if the power supplies and monitors were plugged in.

It seems the modifications made to keyboards to prevent this kind of sniffing has either been removed to save cost or was never tested properly in the first place.

So be careful! If what you are doing is super sensitive you might be better off using an on-screen keyboard.

Source: The Register


Posted in: Hardware Hacking, Password Cracking

Tags: , , , , , ,

Posted in: Hardware Hacking, Password Cracking | Add a Comment
Recent in Hardware Hacking:
- Intel Hidden Management Engine – x86 Security Risk?
- Fitbit Vulnerability Means Your Tracker Could Spread Malware
- Kid Gets Arrested For Building A Clock – World Goes NUTS

Related Posts:

Most Read in Hardware Hacking:
- Elevator/Lift Hacking !!!!! - 79,344 views
- Military Communications Hacking – Script Kiddy Style - 49,822 views
- Hackers Crack London Tube Oyster Card - 45,095 views

Malwarebytes Anti-Exploit Premium | 1 Year 1 PC for $24.95


lm2ntcrack – Microsoft Windows NT Hash Cracker (MD4 -LM)

Your website & network are Hackable


We have covered quite a lot of Password Cracking tools and it’s not often a new one comes out, this one is for quite a specialised purpose (not a general all-purpose password cracker like John the Ripper or Cain & Abel), although you do need to use it alongside JTR.

This tool is for instantly cracking the Microsoft Windows NT Hash (MD4) when the LM Password is already known, you might be familiar with LM Cracking tools such as LCP.

The main problem is you’ve got the LM password, but it’s in UPPERCASE because LM hashes are not case sensitive, so you need to find the actual password for the account.

Example : Password cracker output for “Administrator” account

  • LM password is ADMINISTRAT0R.
  • NT password is ?????????????.

We aren’t lucky because the case-sensitive password isn’t “administrat0r” or “Administrat0r”. So you cannot use this to connect to the audited Windows system.

This password contains 13 characters but launching my password cracker on the NT hash is a waste of time and there is a poor chance of success.


Note :

  • Password length : 13 characters.
  • Details : 1 number + 12 case-sensitives letters.
  • Possibilities : 2^12 = 4096 choices.

In this example, lm2ntcrack will generate the 4096 possibilities for the password ADMINISTRAT0R and, for each one, the associated NT MD4 hash. Then, search for matching with the dumped hash.

Execution time : < 2 seconds to crack more than 1200 NT Hashes. You can download lm2ntcrack here: lm2ntcrack-current.tgz

Or read more here.


Posted in: Hacking Tools, Password Cracking, Windows Hacking

Tags: , , , , , , , , ,

Posted in: Hacking Tools, Password Cracking, Windows Hacking | Add a Comment
Recent in Hacking Tools:
- PowerOPS – PowerShell Runspace Portable Post Exploitation Tool
- Shadow Brokers NSA Hack Leaks 0-day Vulnerabilities
- UFONet – Open Redirect DDoS Tool

Related Posts:

Most Read in Hacking Tools:
- Top 15 Security/Hacking Tools & Utilities - 1,987,183 views
- Brutus Password Cracker – Download brutus-aet2.zip AET2 - 1,456,961 views
- wwwhack 1.9 – Download wwwhack19.zip Web Hacking Tool - 684,170 views

Malwarebytes Anti-Exploit Premium | 1 Year 1 PC for $24.95


DarkMarket Carding (Credit Card Fraud) Site Part of FBI Sting

Your website & network are Hackable


You may remember the story about the Pro ATM Hacker ‘Chao’ and his Tips a while back, apparently that was the start of a big global sting operation on credit card fraud.

Chao was admin/moderator on a community of carders (where they bought/sold stolen credit card info) called DarkMarket and the first to be busted, it turns out the site was actually ran by an FBI agent (J. Keith Mularski AKA Master Splynter) and operated by the National Cyber Forensics Training Alliance (NCFTA)

There have been many reports this week that fraud website DarkMarket, which closed earlier this month, was shut down because of an international intelligence operation, which has lead to up to 60 people being arrested.

All is not as it seems, however, as CNET is stating that the website was all part of an FBI sting operation. It all stems from an article in Wired, where they uncovered that the webmaster of the site was non other than J. Keith Mularski, a senior FBI agent, going under the guise of Master Splynter.

Splynter wrote on the site, just before its demise: “I guess it was only time before this would happen. It is very unfortunate that we have come to this situation, because … we have established DM as the premier English speaking forum for conducting business. Such is life. When you are on top, people try to bring you down.”

There have been multiple arrests in UK following the sting, it seems like this was a very long and rather successful operation taking down a number of key players.

60 arrests is a pretty decent number, each of them must have perpetrated quite a lot of fraud so you can imagine the money being saved with these guys behind bars.

The elaborate ruse is backed up by information uncovered by Südwestrundfunk, a German radio network, who found out the site was being operated by the National Cyber Forensics Training Alliance (NCFTA).

Sting or not, the closing down of the website has lead to 60 arrests, with one person charged with purchasing £250,000 pounds’ worth of stolen data in under two months.

Recent arrests in the UK have so far been made in Leicester, Manchester, Humberside, South Yorkshire and London. DarkMarket administrator Cagatay Evyapan, aka Cha0, was arrested by Turkish police at the beginning of September.

That’s a lot of stolen cards, I wonder how they even get all this information? I guess they have multiple ways with online info being exposed…a lot of the hacks don’t even get published, just covered up.

Then oldskool physical methods like tagging/cloning in petrol stations and restaurants, rigging ATM machines and just plain stealing the cards from the mail.

Source: Techradar (Thanks Navin!)


Posted in: Legal Issues, Privacy, Spammers & Scammers

Tags: , , , , , ,

Posted in: Legal Issues, Privacy, Spammers & Scammers | Add a Comment
Recent in Legal Issues:
- The Panama Papers Leak – What You Need To Know
- FBI Backed Off Apple In iPhone Cracking Case
- TalkTalk Hack – Breach WAS Serious & Disclosed Bank Details

Related Posts:

Most Read in Legal Issues:
- Class President Hacks School Grades - 80,715 views
- Hospital Hacker GhostExodus Owns Himself – Arrested - 47,651 views
- One Of The World’s Most Prolific Music Piracy Groups Busted - 43,629 views

Malwarebytes Anti-Exploit Premium | 1 Year 1 PC for $24.95


Web-Harvest – Web Data Extraction Tool

Your website & network are Hackable


Web-Harvest is Open Source Web Data Extraction tool written in Java. It offers a way to collect desired Web pages and extract useful data from them. In order to do that, it leverages well established techniques and technologies for text/xml manipulation such as XSLT, XQuery and Regular Expressions. Web-Harvest mainly focuses on HTML/XML based web sites which still make vast majority of the Web content. On the other hand, it could be easily supplemented by custom Java libraries in order to augment its extraction capabilities.

Process of extracting data from Web pages is also referred as Web Scraping or Web Data Mining. World Wide Web, as the largest database, often contains various data that we would like to consume for our needs. The problem is that this data is in most cases mixed together with formatting code – that way making human-friendly, but not machine-friendly content. Doing manual copy-paste is error prone, tedious and sometimes even impossible. Web software designers usually discuss how to make clean separation between content and style, using various frameworks and design patterns in order to achieve that. Anyway, some kind of merge occurs usually at the server side, so that the bunch of HTML is delivered to the web client.


Every Web site and every Web page is composed using some logic. It is therefore needed to describe reverse process – how to fetch desired data from the mixed content. Every extraction procedure in Web-Harvest is user-defined through XML-based configuration files. Each configuration file describes sequence of processors executing some common task in order to accomplish the final goal. Processors execute in the form of pipeline. Thus, the output of one processor execution is input to another one. This can be best explained using the simple configuration fragment:

When Web-Harvest executes this part of configuration, the following steps occur:

  1. http processor downloads content from the specified URL.
  2. html-to-xml processor cleans up that HTML producing XHTML content.
  3. xpath processor searches specific links in XHTML from previous step giving URL sequence as a result.

Web-Harvest supports a set of useful processors for variable manipulation, conditional branching, looping, functions, file operations, HTML and XML processing, exception handling. See User manual for technical description of provided processors.

You can download Web-Harvest 1.0 here:

webharvest1-exe.zip

Or read more here.


Posted in: General Hacking, Hacking Tools, Web Hacking

Tags: , , , , , , ,

Posted in: General Hacking, Hacking Tools, Web Hacking | Add a Comment
Recent in General Hacking:
- BADLOCK – Are ‘Branded’ Exploits Going Too Far?
- Dradis – Reporting Platform For IT Security Professionals
- Kid Gets Arrested For Building A Clock – World Goes NUTS

Related Posts:

Most Read in General Hacking:
- 10 Best Security Live CD Distros (Pen-Test, Forensics & Recovery) - 1,170,877 views
- Hack Tools/Exploits - 628,831 views
- Password Cracking with Rainbowcrack and Rainbow Tables - 435,488 views

Malwarebytes Anti-Exploit Premium | 1 Year 1 PC for $24.95