Analysis: Search Engines Are Not Created Equal

The pay-per-click test Practical eCommerce conducted on search engines Google, Yahoo!, Ask and MSN netted some interesting data, but perhaps the most valuable information came outside the numbers.

I have never set up this type of pay-per-click (PPC) campaign, so I came to this project without any preconceived ideas of what worked best.

I concede that any of the multitude of ecommerce owners conducting a pay-per-click campaign might have a different experience than I have had, but I wanted to outline some of the challenges and discoveries I’ve experienced during this project. Perhaps the lessons noted below will be helpful as you make decisions on whether to launch a pay-per-click campaign and what search engine(s) you might use.

Lesson No. 1: It was easier to set up ads on MSN.

I had a great first impression of MSN. I found its navigation easy to understand and efficient. The process to set up my two listings was easy. (My experience with MSN went poorly from there — see below).

In the beginning, I found the other three a bit more clunky to navigate. However, after learning the various nuances of Google, Ask and Yahoo!, I became more comfortable with each one.

Lesson No. 2: I could start my campaign more quickly with Google and Ask.

With both Google and Ask, my campaigns started on time and without any kind of lengthy “editorial review.” Yahoo! requires a listing to go into an “editorial review” process that typically takes three business days from the time a listing is submitted. So, if you submit a Yahoo! campaign on Saturday, it most likely couldn’t begin until Wednesday. This is a very frustrating feature.

Yahoo! outlines the purpose of the review in its email communication to the customer:

“The review will check your site content, titles and descriptions for relevancy to each requested keyword, and to ensure that your site complies with our listing guidelines. All listing requests must undergo an editorial review to ensure that our users consistently have a high quality search experience, which in turn, will bring more targeted leads to your site.”

Maybe that’s true. In my case, I didn’t need a review. I knew what keywords I wanted to use to ensure comparable results across four search engines. I was also using keywords and ads from two companies that had already used these ads in previous pay-per-click campaigns. I felt comfortable these ads would work.

So, even if you have your ad professionally written and vetted by a pay-per-click consultant, you’ll go through a multi-day review process at Yahoo! right now (apparently, this will be changing soon, according to the company).

Yahoo! did send me emails as each keyword was approved in their process, and that was a handy way to keep up with their progress. MSN noted submissions to its site also go through a review, but the process was apparently much quicker. However, MSN doesn’t send an email alerting me I’ve been approved, so I have no idea how much time that actually took.

MSN states it does that to minimize the email communication a customer receives. However, I would much rather have the communication rather than waiting in limbo or having to go back to the site periodically to see if my listing was approved. Unfortunately, on the MSN site, it doesn’t indicate whether your ad is “approved.” The status update only listed my ads as “submitted.” In one of my conversations with a MSN customer service representative, she told me they are working on making that status more clear to the customer.

Lesson No. 3: Google’s campaign management tools topped the other search engines.

Google’s advanced management functions provided me the option to schedule the ads to run on the days of the week I desired and at specific times of the day. I could also schedule the ads to run in the countries of my choosing and they could appear in the language I desired.

MSN’s management tools were very similar in function to Google’s, but apparently the tools didn’t completely work. I had selected “worldwide distribution” for my ads and, apparently, the appeared only in Canada (see Lesson 6 below for more details).

Ask provided a basic start/stop function, but few of the advanced tools offered by Google.

Yahoo! didn’t allow even a basic start/stop function. Upon approval by Yahoo!, the ad went “live.” I could suspend the ad when I wanted it to stop running, but I never found any automated functions. In fact, since I wanted my test campaign to begin at midnight, I had to get up at midnight to activate the Yahoo! ad manually.

Lesson No. 4: Yahoo!, Google and MSN each had decent reporting functions.

Google and MSN get a slight nod over Yahoo! based on ease of use, but these three were relatively close in reporting functions. Each of these provided the basic information (average position listing, cost per click, budget used, etc.). The data from Ask, however, was unclear as of press time. There was about a five-day lag in reporting so we couldn’t get a full accounting of our results. Not good.

Lesson No. 5: I know things don’t always go smoothly, so having a problem-solving team in customer service is critical.

Anyone who has been in business knows that sometimes a customer does not have a good experience. However, customer relationships can be salvaged with great customer service.

I did not have a need to talk with customer service at Google or Ask as both of the campaigns on those sites activated without a hitch. However, that was not my experience with Yahoo! and MSN.

As I noted above, Yahoo! currently goes through a review process before activating a listing. Since the review process can take up to five days after posting the ad, the wait can be frustrating for an ecommerce owner who wants to see his ad in the marketplace immediately. I contacted two people at Yahoo! Search Marketing about expediting the posting of my listing and bypassing their review. Both were adamant that couldn’t happen and explained Yahoo! would have a new product on the market soon that would allow for a quicker posting. The customer service folks at Yahoo! were very pleasant, but they couldn’t give me what I needed.

There was a significant problem with our listings at MSN. I don’t know if this is a frequent problem for the company, but was certainly an issue for me on July 13. I was online at midnight to ensure my listings activated on time. I could not find my listing on MSN. I went through page after page of organic listings, and my ad wasn’t there. I waited a couple of hours and still couldn’t find it. MSN’s customer service office isn’t open at 2 a.m. (and I didn’t expect it to be, frankly), so there was no way to get a quick resolution. I checked for the ad again at 5 a.m. — still not there.

The customer service office for MSN opens at 6 a.m. Pacific Time. Over the course of the morning, I had five discussions with four different customer service representatives at the company. I have to say that each person was very pleasant and understanding of my problem. The company’s “complaint ticket” process allowed each person to quickly review conversations I’d had with other reps. At one point, I was on the telephone with one of the reps and she was telling me she could see my listing. However, I could not. She emailed me a screen shot to verify we had the top position at that point in the day but, through that process, we discovered our ad was only visible using the MSN search in Canada. It was not visible in the United States. That was very disappointing to find out.

The customer service representatives could not explain that technical glitch (as it was clearly noted on my account that I had selected worldwide distribution), so my issue was transferred to the “development team” for review. As of press time, four days after my complaint went to the technology team, I don’t have an answer as to why there was this major glitch for MSN.

Despite the problem, I’d have to give high marks to the customer service team at MSN for its helpfulness and, since I still don’t have an answer, a low mark to the development team for lack of a timely response. Either the MSN development team is too understaffed to handle customer inquiries quickly or they are swamped with too many problems to fix.

I found the process of setting up, watching and troubleshooting multiple accounts a time-consuming and exhausting process.

I don’t doubt PPC can be a successful tool for certain businesses. There are many pay-per-click and pay-per-call experts in the marketplace with which an ecommerce business can consult about a campaign to help maximize its success. PPC can be expensive and you must keep in mind the “big picture” when conducting a campaign — the cost to attract a customer is often expensive but, if you can keep that customer, his value to your bottom line is significant over the long term.

Under most circumstances, I would not advise a small ecommerce business to spend its entire advertising budget on PPC options. If you’re not carefully watching the program, you can find yourself spending a lot of money and still buried on page 3 of the sponsored ads.