Fullstack Lessons: JavaScripting Craigslist for Fun and Profit

In today’s online lesson, we’re going to teach you how to write some code that collects email addresses from a Craigslist search results page. You’ll be able to run it right in your browser’s console, without having to set up separate files. Let’s get started!

Background
If you’re taking this lesson then you’ve probably used Craigslist before to buy (or try to buy) something. And you probably know it can be a tedious process; lots of repeated clicking and emailing. As a coder, whenever you encounter a situation like this you should think “There has to be a better way”… And there is.

The Problem
When writing code to solve a pain point, you always want to think about the problem first. In this case it’s simple: It takes too long to go through a list of results from a search on Craigslist and to email each seller.

The Solution
Ideally, our solution will be easy to implement. Thinking about navigating Craigslist, after you search for something you usually click on each result, find the email address of the seller, and send an email to them. Our coding solution will do this for us.

Since we want our code to be easy to implement and we know it has to interact directly with the Craigslist website, we’ll use JavaScript as our programming language. We’ll also make sure the code can execute right in our web browser’s console.

The Console
Assuming you’re using Google Chrome, you can bring up the console by pressing Command-Option-J (Mac) or Control-Shift-J (Windows/Linux). If needed, you can read more about the Chrome console here. The console gives you a live shell prompt where you can run JavaScript and interact with the current web page you’re on. Firefox also has a web console.

The Code
Head over to craigslist and search for something you’d be interested in buying. Make sure you narrow your search to a nearby location like a city (e.g. New York City, or Manhattan). And ideally target your search with price range, image-only, descriptive keywords, etc.

Once you’ve got your search results page loaded, bring up the browser console and paste in this code:

var emails = "";
$(".txt").find("a.hdrlnk").map(function(i, el) { return el })
.each(function(i, el) { 
 $.get(el.href, function(body) { 
 $.get( $(body).find("#replylink")[0].href, function(linkbody) {
 var email = $(linkbody).find('a[href^="mailto:"]')[0].innerText + ',';
 emails += email;
 })
 })
})
setTimeout(function() { 
 console.log('emails: ' + emails);
}, 5000)

At this point your browser should look similar to this:

If you hit enter in your console to execute your code, after a few seconds you should see a list of email addresses printed in the console! You can now copy and paste that into an email “To” or “BCC” field and it will be addressed to each one.

Think of possibilities of just this one small coding “hack”– you can now turn the tables on Craigslist and get sellers working to respond to you. Whether it’s for finding a new couch, or applying to jobs… You’ve just harnessed the power of JavaScript to make your life easier.

Now let’s walk through how our code works..

Code Walkthrough

Steps
Assuming we’re starting to code from scratch, we have to ask ourselves: What will our code actually do? Let’s assume, like in the example, you’re looking to buy a couch on Craigslist. After running a search, we’ll have our code to go through each result on the page, find the email address for the seller, and return a list of all those email addresses to us. Once we have that list, we (not the code) can draft up one email and send it out to all the email addresses.

So to execute this, from a Craigslist search results page, we’ll need to:

1. Simulate a “click” on each link on the page
2. For each “click”, we’ll receive back the web page for that specific listing. From there, we need to simulate a “click” on the “Reply” button
3. After receiving the result of hitting that “Reply” button, find the email address within that data
4. Store each email address in a running list of email addresses
5. Finally, Return the list of all email addresses

Code
Now for the exciting part– The actual code!

Let’s execute our steps line by line. We’ll be using the popular JavaScript library jQuery to help us easily interact with the Craiglist web pages.

Step 1 - Create running list of email addresses

var emails = "";

Explanation: We define a variable named emails, and make it an empty string. As we collect emails later we’ll add each to this string.

Step 2 - Find all listing links on the search results page

$(".txt").find("a.hdrlnk").map(function(i, el) { return el })

Explanation: $(“txt”) uses jQuery to select all the html text on a page. The .find(“a.hdrlnk”) uses the jQuery find method to search the selected html text for hyperlinks (“a”) with the class ‘hdrlnk’. The find method returns a jQuery collection of elements, in this case links, which we then run the map method on to create a normal JavaScript array of those elements.

So at this point our code outputs an array of html link elements for all the listings on our search page.

Step 3 - Simulate “clicking” each link by making a browser like ‘GET’ request

.each(function(i, el) { 
 $.get(el.href, function(body) {

Explanation: We use the each method to iterate over html element in the array we got from Step 2, and for every element we use the jQuery $.get method to simulate clicking the link (this actually makes an HTTP GET request). We pass the method the actual url for each element, ‘el.href’, and callback function that takes an argument body, which will be the html from of the page we just “clicked” with our code.

Step 4 - “Click” the “Reply” button on each listing web page

$.get( $(body).find("#replylink")[0].href, function(linkbody) {

Explanation: We use the $.get method again to click the Reply button link on the page we received from a simulated click in Step 3. The first argument for the $.get method, $(body).find(“#replylink”)[0].href, uses the find method to find the URL of reply button link on that page. The 2nd argument is a callback function which takes the resulting html from that Reply button click. This is the html that has the seller’s email address, so we know we’re almost there. And remember, this is all happening for each link on the search page, from us using the .each method iterator in Step 3.

So now we have linkbody, which is some html containing our target email address..

Step 5 - Find the email address and add it to the running list of emails

 var email = $(linkbody).find('a[href^="mailto:"]')[0].innerText + ',';
 emails += email; 
 });
 });
});

Explanation: The first line our code here finds the email address in the ‘linkbody’ html, and assigns it to a variable ‘email’. The second line adds that email address to the string ‘emails’ we created in Step 1. Since this step is happening for each listing on the results page, each listing’s email address will be added to the ‘emails’ string. The remaining brackets and parenthesis close out the functions we opened up in Steps 2-4.

Step 6 (Final) - Print out all emails!

setTimeout(function() { 
 console.log('emails: ' + emails);
 }, 5000)

Explanation: Here we use the setTimeout function to create an artificial delay of 5000 milliseconds (5 seconds), which will give our code from steps 1-5 time to execute and add all the email addresses from the page’s listings to the string emails. Then we use console.log to print out the emails. And we’re done.

All together the code is only 13 lines of JavaScript:

var emails = "";
$(".txt").find("a.hdrlnk").map(function(i, el) { return el })
.each(function(i, el) { 
 $.get(el.href, function(body) { 
 $.get( $(body).find("#replylink")[0].href, function(linkbody) {
 var email = $(linkbody)..find('a[href^="mailto:"]')[0].innerText + ',';
 emails += email;
 })
 })
})
setTimeout(function() { 
 console.log('emails: ' + emails);
}, 5000)

And there you have it - a JavaScript code snippet that is both simple and powerful… Now you can use your new code any time you’re conducting a large Craigslist search!

Next Steps
Hopefully your new Craigslist “hack” has helped you see the power of code. In reality the code we wrote here was pretty basic. If you’re just starting out learning to code, check out these freeresources to better understand what happened in this Craigslist “hack”:

1. Understanding JavaScript: Codecademy JavaScript

2. Learning jQuery: Codecademy jQuery

Now imagine what you could do with the knowledge of a full fledged programmer….

If you’re interested in taking your coding skills to the next level and becoming a developer, check out Fullstack Academy and download our comprehensive “Road to Code” starter guide.