Off-topic Talk Where overpaid, underworked S2000 owners waste the worst part of their days before the drive home. This forum is for general chit chat and discussions not covered by the other off-topic forums.

Any programmers in the house?

Thread Tools
 
Old Apr 28, 2009 | 06:43 PM
  #1  
Skuzzy's Avatar
Thread Starter
Registered User
 
Joined: Sep 2002
Posts: 11,644
Likes: 0
From: panhandle state
Default Any programmers in the house?

I want to write a computer program that would scan a set of webpages and find predetermined criteria. Example: a program that searches all Dallas craigslist listings containing "honda civic" and records the percentage of those listings which contain "fast and furious".

What language should I look into learning?

Or, frankly, do you feel that such an undertaking is ludicrously ambitious for someone with my programming experience (very little, years ago)?
Reply
Old Apr 28, 2009 | 10:56 PM
  #2  
rustywave's Avatar
Registered User
 
Joined: Feb 2005
Posts: 3,605
Likes: 0
From: Chicago
Default

i haven't done it, but it doesn't seem to be too difficult. at least the searching part. i'm not too familiar w/web stuff so i'm not sure how you would traverse every available listing.

however, if you were to get that list...you could probably use any language to read the page in as text, then some sort of instr() function to see if it has your desired text. might be a little slow, though...haha. sorry good luck.
Reply
Old Apr 29, 2009 | 05:54 AM
  #3  
JonBoy's Avatar
20 Year Member
Liked
Loved
Community Favorite
 
Joined: May 2002
Posts: 19,734
Likes: 247
Default

Use google?
Reply
Old Apr 29, 2009 | 06:02 AM
  #4  
thebig33tuna's Avatar
 
Joined: Jan 2007
Posts: 32,283
Likes: 0
From: Cincinnati, OH
Default

so you're talking about a web crawler going to sites and picking up page statistics? you need to write yourself a web crawler first, then you need to store your website data (whatever you're collecting) in some form of simple dB or file structure... not ridiculously complicated but just offhand i don't know how i'd go about it. i wrote a simple word search program that would index all the words in a text file and give you a word count on them back in college, that part is pretty simple. the web crawler is a bit more complicated. i've never written something like that.

http://en.wikipedia.org/wiki/Web_crawler

as for what language, i dunno. there's plenty of options. download a freebie version of visual studio or download one of the many free java IDEs. mess around. google is your friend as far as learning the languages and figuring out what free code is out there that you might be able to use.
Reply
Old Apr 29, 2009 | 07:29 AM
  #5  
DFWs2k's Avatar
Registered User
 
Joined: Oct 2005
Posts: 14,614
Likes: 2
From: Denton, TX
Default

Originally Posted by thebig33tuna,Apr 29 2009, 08:02 AM
... google is your friend as far as learning the languages and figuring out what free code is out there that you might be able to use.
I agree, I don't think the language you choose really matters
Reply
Related Topics
Thread
Thread Starter
Forum
Replies
Last Post
airgate
Off-topic Talk
11
May 10, 2002 09:12 AM
ironwedge
Off-topic Talk
2
Dec 19, 2001 08:54 PM




All times are GMT -8. The time now is 04:53 AM.