surrsea2.htm: Fravia's how to search the web, Surreal5's second lesson

part 2: how to find Warez/mp3's/xxxpwords etc

This goes a little beyond just using search engines. I assume you have a
basic knowledge of FTP, and know how to use search engines. I'll use warez
as example, but the same search strategy goes for mp3's, passwords, and
anything with a time sensitive character, like cool live realaudio surf
soundtracks ;)

FTP search sucks, forget it as your main warez finder, i use it only to
locate missing files. Its noise/signal ratio is bad, even when you know
what day(s) of the week it is updated. The best way to find things with a
time sensitive character is to monitor the pages of the uploaders. For this
you will need a program that monitors webpages and notifies you
automatically when they are updated. I use Tierra Highlights2, because it
has an intuitive interface, and works as a toolbar. An alternative would be
Netattache, but i have bad experiences with it. If you already use a page
monitor, with at least a 15 minute update interval, by all means keep on
using that, but i'm using highlights as an example. 


Step 1: Setting up Highlights.

Highglights comes in a crippled demo version, it's time limited and has a
15 page maximum. Datecracker removes the time limit, but there is NO crack
for the 15 page maximum, at the time i write this . (believe me i've looked
for it). The program is cheap (around 35$) considering the good it can do
you. 

With Highlights you add a page (much like a bookmark), except it monitors
the page for you. It retrieves the page every 15 minutes (or to whatever
you set it) and highlights the changes in a different colour. In other
words within 15 minutes after uploader X, tells the world (on his page)
where he has uploaded to, you'll know about it, and are one of the first
into the ftp if you want a certain proggie. Considering warez stay on a
public ftp for about a day or two, knowing it was added only maximum 15
minutes ago, makes it worthwile checking out. Over 90% of the links i
follow this way, i get in fast, and the warez are there. Ofcourse this
ratio depends highly on the quality of the pages you monitor. So you need
to find and maintain a list of good pages.


Step 2: How to find good pages.

This is where your search technique comes in again. If you have to start
from scratch, there are two ways to find warez pages, wether they are good
takes a bit of trial and error.

1) go visit the warez top's, forget the first 10 or so, and start following
links.
2) use a search engine with a date filter and look for warez pages added in
the last few days, or look
for programs that you need :)

Go by these rules when you need to judge wether a warez page is suitable
for monitoring:
- warez pages without warez are not warez pages ;)
- warez pages that were updated longer than one or two days ago, are of
little use
- warez pages that do not contain the page owners own uploads, are of
little use. (if you start from scratch, you may keep a few leech pages
though, an update on a leech page, does mean that he got in recently...) 
- Wether the site has direct links or ftp adresses makes no difference, as
long as it's updated, and the content of what's on the ftp is shown. Ftp
links without content, are a sure giveaway that the page owner doesn't
upload himself, multiple (more than 2) direct download links for the same
proggie are a sure giveaway that he does.
- Ignore pages that are slow to load, it may be because they get too many
visitors, and your 15 minute advantage becomes useless. For the same reason
you can ignore the pro's, the pro's pages suck, everyone is on there. Go
for the kids pages, they're only good until their enthousiasm runs out, but
boy do they have enthousiasm!!! ;)


Step 3: Maintaining the quality of your list of pages.

A warez page is like a restaurant, as long as there are few people who know
about it, "the food" is good, when there are too many "customers" the
quality and service go down. When there's nobody, it closes. It takes a bit
of experience to recognise the "type" of a warez page and its owner, but
you'll soon know which pages to keep and which to throw out. A few golden
rules:

- page maintainers need feedback for their enthousiasm, click on their
sponsors, or give them a pat on the back, but stay away from that top ...
button!!!
- If you use the demo with 15 page max, always keep one or two 
free for new pages you may run into. It's continuously keeping the best,
and throwing out the ones who's page title you haven't seen for a while.
Remember your search for new pages never ends, when you have a 99.8% succes
ratio, there's always that 99.9%. Ignore it for too long, and your ratio
will fall, or the pages will have moved or are dead, and you can start all
over.
- Ignore the sort by date ranking for making a decision on which pages to
throw out, remember time is not the same on the other side of the globe ;)
Keep the ones that update around the time you are generally online.
- Remember it's better to keep a page of a busy uploader, who doesn't
update every day, than a page of a leecher who updates every hour.
- Throw out design wizzards who update their page without adding anything,
and pages with rotating banners.

For mp3's it's much the same, for passwords i imagine you have an extra use
for highlights by setting the keyword filter to the xxxsite(s) you're
looking for. The datecraked demo should work fine with the 15 pages max for
any of these tasks, but not for all three tasks at the same time, you can
get around it, by using multiple highlights.haf files and moving them to
your dir when you need them, but is the fuss really worth the 35$ i wonder... 
  
have fun...

surreal5.
You are deep inside fravia's page of reverse engineering, choose your way out:
Back to Surreal5's search tips
Back to how to search
Is reverse engineering legal?