WIRED Staff

Scraping MP3 Blogs

Designer Jeff Veen’s instructions for foraging MP3 blogs for new music with Wget, a Unix/OS X/Linux command line site scraper. Veen writes: Wget to the rescue. It’s a utility for unix/linux/etc. that goes and gets stuff from Web and FTP servers — kind of like a browser but without actually displaying what it downloads. And […]

Designer Jeff Veen's instructions for foraging MP3 blogs for new music with Wget, a Unix/OS X/Linux command line site scraper.

Veen writes:

Wget to the rescue. It's a utility for unix/linux/etc. that goes and gets stuff from Web and FTP servers -- kind of like a browser but without actually displaying what it downloads. And since it's one of those awesomely configurable command line programs, there is very little it can't do. So I run wget, give it the URLs to those mp3 blogs, and let it scrape all the new audio files it finds. Then I have it keep doing that on a daily basis, save everything into a big directory, and have a virtual radio station of hand-filtered new music. Neat.

The process is actually more involved and much as I'd like to do this, it's all a bit too complicated. I'll hold out for someone to whip up an OS X utility with a pretty interface.