It’s a question that paid search marketers who seek better performance should ask themselves more often. And who among us doesn’t want better performance?

Unfortunately, a lot of paid search campaigns suffer from a serious keyword bidding problem that most people don’t know about because it hides deep within their campaigns.

This video below will give you the tools you need to find and fix what ails your campaigns.

In just 50 minutes you’ll learn:

• The metrics you need to watch
• What to look for in each
• Strategies to tackle the problem when you find it

Don’t let a keyword bidding problem rob you of revenue and budget.

Do You have a Keyword Bidding Problem? from OptiMine Software on Vimeo.

Tweet

The last excerpt from “Achieving the Gold Standard in Paid-Search Bid Optimization” clearly established that model-based optimization is superior to rules-based and that, in the world of models, global optimization beats local. But there is another layer of of global optimization that needs exploring. Within the global subset of model-based optimization are keyword-level and cluster modeling. The two primary differences between them are how they deal with performance data – especially the sparse data one finds in tail terms – and the financial impact on paid-search programs. If you want to back-story before you dive in here, you can start with “Paid-Search Overview” and “Rules vs. Models”.

Global Keyword vs. Global Cluster Modeling

Global keyword optimization results in improvements that are significantly better than cluster modeling because each keyword is treated individually. Cluster modeling creates groups of keywords, treating each member the same as the others – a hazardous shortcut for advertisers with large, complex paid-search campaigns.

Cluster model-based systems were developed to cope with the “sparse data” problem of tail terms for which there is very little historical data available. As the name implies, clustering aggregates data from several keywords to hundreds and even thousands of keywords, which are assumed to have the same general performance characteristics. Simply put, clustering was developed to assemble data sets large enough to permit human analysts to apply more traditional statistical techniques to determine bids.

While clustering may seem like a necessary way to manage keywords that receive very few clicks or impressions, the technique is in many ways sub-optimal. On the plus side, using clusters leads to model stability, meaning the results are repeatable, which is something statisticians love. But the negative effects of using clustering are substantial. For one, keywords in a cluster might seem similar, but each keyword is actually unique at some level. While statisticians might value the extra data, individual keywords lose their uniqueness in a cluster. The result is a loss of performance compared to modeling each keyword as a unique entity (see figure below), as the advertiser spends too much on some keywords in a cluster and too little on others.

A second issue is clustering typically requires statisticians to tune the models manually. As a result, cluster-based solutions are rarely pure software applications and thus can’t offer the degree of automated bid optimization needed by advertisers managing thousands or millions of keywords in a dynamic bid-based marketplace. A third issue is the time-intensive task of creating clusters, as all statistical models decay over time, making it necessary to periodically retrain the statistical models involved. For these two reasons, cluster models tend to be used far beyond their useful life cycle with a resulting negative impact on performance.

Clustering survives due to the belief that keyword-level modeling, while far superior in performance terms, can’t be done on tail terms for which there simply isn’t enough data available to build accurate models. That’s why most paid-search bid optimization vendors extensively rely on clustering, even though they might not reveal this fact to customers. Why? Clustering has yet to be automated and requires human analysis driving up cost and response times, even as it leads to sup-optimal performance on the vast majority of keywords in a campaign.

Fortunately for advertisers, this belief is false. Keyword-level modeling can be done on keywords with as few as 10 conversions a year. What’s required is the right balance of the appropriate math, software automation, and transparency into the specific variables that drive individual keyword performance. Global keyword-level modeling is the gold standard of bid management. It regularly improves performance (profit, revenue, ROAS, etc.) by 25 percent or more in controlled tests against global cluster-level, local keyword-level and rules-based technologies. The basic elements driving success for global keyword-level modeling can be summarized as follows.

Appropriate math: The mathematical approach to global keyword-level modeling rejects the “requirement” to cluster in order to create sufficient click or impression data. Clustering might produce an acceptable “average” for all keywords in the cluster but has little relevance to the future performance of the individual keywords the cluster contains. This accounts for the sub-optimal performance of clustering versus modeling the specific performance of the individual keywords within the cluster, which is achievable for even keywords with small amounts of data. Achievable, that is, using the appropriate mathematical modeling techniques – such as structural risk minimization, a technique that trains models to become simpler as data sets become sparser – as opposed to complex clustering techniques, which deal with sparse data by building models that are often far more elaborate than the data will support.

In one controlled test, using the right math to individually model each keyword drove 216 percent more account sign-ups than a competing cluster-level solution. One issue in this test was the age of clusters used, which hadn’t been refreshed and contained keyword groupings that were simply obsolete. Keyword-level modeling techniques identified a handful of good keywords hidden in clusters of bad keywords. Separating those out and bidding them up led to the volume increase.

Since keyword-level modeling eliminates the need to cluster, it can be achieved through software automation. For advertisers dealing with large numbers of keywords who want to avoid the performance sacrifice in long-tail keywords that is inevitable with clustering, automation is usually an advantage in a dynamic advertising marketplace, both in terms of lower cost and superior daily bid optimization of all keywords in an SEM program. The same type of software automation techniques can be applied to retraining the models that predict keyword performance. The models can be “taught” to automatically learn based on the most recent inputs that make them much more responsive to changing marketplace conditions.

Transparency: Transparency is the ability to make visible to the advertiser the individual variables that drive keyword performance to build understanding and trust. In this sense, clusters are about as transparent as mud, since the complexity of clusters makes them almost impossible to describe. Modeling keywords individually allows advertisers to make intelligent determinations on each and every keyword in an SEM campaign. They can see the decisions the software made and why. Figure 3 compared five variables that drove performance in two similar but distinct keywords. Using clustering, marketers would be forced to conclude that only one variable – cost per click – was the common and determinant performance factor, whereas in reality, the two keywords have very dissimilar performance profiles and require very different optimization strategies.

Bottom Line

Model-based keyword bid optimization is by far superior to rules-based, but selecting the right model-based system is critical to maximizing paid-search campaign performance. Where local optimization is incrementally better than rules-based, the performance improvement does not approach that of global cluster or global keyword. But only global keyword-level modeling, the gold standard of bid optimization, will truly maximize paid-search performance.

Keyword-level modeling for all keywords in an SEM program, not just the low-hanging fruit of head terms, is a demonstrably superior approach and delivers overall performance gains of 25 percent or more over clustering and other techniques. Software driven bid optimization techniques that use keyword-level modeling also provide advertisers with greater flexibility and control. Clusters take time to create and rebuild in response to changing business goals and marketplace conditions. With automated keyword-level bid optimization techniques, advertisers can continually change campaign goals and constraints and rebid keywords at any frequency they need.

Keyword-level, global bid optimization is the ultimate solution for deriving maximum profit from SEM campaigns that involve a large number of keywords. It’s the right choice for optimizing head terms, tail terms and everything in between.

@OptiMineInc

Tweet

Today we continue excerpting “Achieving the Gold Standard in Paid-Search Bid Optimization”. The last post provided an overview of paid search, setting the table for the discussion of rules-based and model-based methodologies. In addition to these two, we’ll look within model-based optimization at the differences between local and global optimization. What may seem simple on the surface is really significant and dramatic in the financial impact it has on complex paid-search programs. 

Rules vs. Models

The benefits of modeling exceed those of rules-based optimization, which reacts to situations rather than predicting and adjusting by learning historical conversion data.

As shown in the Figure 1 below, there are two main approaches to bid optimization: rule-based versus model-based. A model-based system uses historical performance data to train statistical models to predict future performance. For example, a model-based system could predict the bids necessary to achieve a 200 percent ROAS tomorrow. They contrast with rules-based systems, which use a pre-defined set of reactions to certain situations. For example “if ROAS is less than 200 percent, then lower bids by 10 percent.” In general, models-based systems are predictive while rules-based systems are reactive.

Global Optimization vs. Local Optimization

Global optimization strives for the success of the whole campaign, while local optimization makes sure every keyword achieves the same goal, regardless of the overall impact.

Within model-based approaches, we need to make a distinction between local and global optimization (See Figure 2). Local optimization simply bids each keyword separately from the other keywords in an SEM program. For example, if your target ROAS is 200 percent, then every keyword is bid to obtain an individual ROAS of 200 percent. A local solution won’t trade off a low ROAS on one keyword with a high ROAS from another.

FIGURE 2: In this simple example there are two keywords (A & B) that only have two possible bids each (High & Low), for a total of four possible bidding scenarios. The goal is maximizing orders; the constraint is a $60 maximum cost per order. With local optimization, all keywords must meet the constraint. With global optimization, all keywords are considered at once and bid individually, leading to a substantially lower cost per order and a dramatically higher ROAS.

Global optimization (referred to as a portfolio approach by some vendors) considers all of the keywords at once, assigning bids so that, on average, the group as a whole maximizes a goal while meeting certain constraints, such as a specific ROAS or minimizing cost per action (CPA). The advantage of global optimization is that it treats each keyword appropriately with respect to the whole campaign. For example, it might turn out that one keyword can drive significant revenue at a ROAS of 180 percent, while another drives the same amount of revenue at a ROAS of 220 percent. As long as the average ROAS is 200 percent, the global solution will declare success. This approach generally provides higher value from a set of keywords than local optimization.

Having been introduced to Global Optimization, you are now ready to dive into the two specific types of modeling  that exist within it; keyword-level and cluster-level modeling. As with Global vs. Local, the financial improvement on paid-search programs that use keyword-level modeling is tremendous. If you don’t want to wait, download “Achieving the Gold Standard in Paid-Search Bid Optimization” (no registration required).  

@OptiMineInc

Tweet

“Achieving the Gold Standard in Paid-Search Bid Optimization” was the first white paper published by OptiMine and remains the most widely distributed. One of the reasons for its popularity is the way it distills the complex world of bid optimization methodologies into an easy-to-understand guide. Whether you are a paid-search veteran, or just starting out, the Gold Standard white paper will help you segment and understand the differences in approach and results among the various optimization techniques. Today we embark on a series of posts that excerpt Achieving the Gold Standard. We begin with an overview of paid search. 

Introduction
Paid-search advertisers have a range of choices when selecting a keyword-bidding approach: rules-based bidding and three different versions of model-based solutions. This paper will analyze the differences in these bidding techniques and explain why global keyword-level bid optimization delivers performance gains of 25 percent to 200 percent more than what can be achieved with the other bidding approaches.

Paid-Search Overview
Google’s advertising model is both simple and fiendishly complex. In most cases, paid-search advertisers bid on keywords daily to locate their text ads in the most advantageous positions on Google’s search engine. The bid, combined with Google’s Quality Score—a formula Google has developed that factors in an ad’s quality, relevance and historical click-through performance—determines an ad’s
position.

In effect, advertisers have to predict the future cost and value of every single keyword in their search engine marketing campaigns. It’s not a trivial task. Large paid-search advertisers may need to optimize tens of thousands to millions of keywords to meet a variety of different campaign objectives. Not all keywords have the same weight, of course. Head terms, or keywords that trigger a large number of clicks and conversions, are usually the focus of most SEM efforts. Tail terms, or keywords that might receive just a few clicks per day or week, receive less attention. But regardless of where a keyword lies on the distribution, if its bid is too low, sales are lost. If the bid is too high, more money is spent on that keyword than its potential return or value merits. Achieving the best performance takes automation, science and domain knowledge.

Over the past few years, search engine management consultants and digital marketing platform vendors have developed a variety of products and services to help advertisers optimize their paid-search spend. In most cases, the efforts involve some combination of human analysis and model-based bid optimization software, sometimes called an autobidder. Technology is essential because the scale of the challenge easily outstrips human analysis alone, especially when it comes to tail terms, which have little click history and which make up the vast majority of keywords.

A common perception among paid-search advertisers is that model-based bid optimization software is something of a black-box technology that is only appropriate for tail terms and that advertisers should “manually optimize the important head terms of an account.” This perception is based on some of the early solutions in the marketplace, which did indeed keep their inner workings secret and typically delivered only modest short-term performance gains that tended to taper off over time.

Today, however, bid optimization technology has developed to the point that it can maximize financial performance across all keywords while being  transparent and meeting critical business constraints – for example, maximizing profit while providing some minimum order volume, or maximizing revenue while maintaining a minimum return on ad spend (ROAS). For advertisers managing a large number of keywords, some form of model-based bid optimization technique is virtually essential. But success in using bid optimization software requires understanding the advantages and limitations of the different modelbased techniques and the recognition that there are no set-and-forget solutions. And bid optimization still requires some degree of human intervention from analysts with domain knowledge and an understanding of the business.

The next excerpt will look at the differences between rules-based and model-based optimization methods and, within models, how local optimization differs from global optimization. “Achieving the Gold Standard in Paid Search Bid Optimization” can be downloaded now. 

@OptiMineInc

Tweet

A few months ago OptiMine published its first white paper, “Achieving the Gold Standard in Paid Search Bid Optimization”. In it, OptiMine CTO Rob Cooley made the case that using global keyword-level modeling to optimize keyword bidding is superior to other modeling techniques a that employ clusters. The reason, quite simply, is that keyword-level modeling treats each and every one individually, regardless of the number of keywords.

At the end of the white paper a simple question was implied: Why cluster keyword data if you don’t have to?

The answer, also implied: There’s no reason to. 

On June 5th and 6th the only bid optimization system that models every keyword individually will be in booth 24 at SMX Advanced. We invite you to stop and see why OptiMine guarantees paid search financial performance improvements of 25% or more. Yes, it’s all in how we treat the keywords – as individuals. While you’re there, we also invite you visit our competitors booths. Compare OptiMine results to those that use clustering, local, and rules-based optimization methods.

If you can’t make it to Seattle, the OptiMine website lays out the case quite nicely in this video and in several case studies.

Tweet

Paid-search bid optimization comes in two flavors: rules-based and model-based. Within the broad realm of model-based optimization, you’ll find three common methods: global cluster-level modeling, local keyword-level modeling and global keyword-level modeling. Each has pros and cons and each offers various degrees of performance improvement.

If you’re not certain what method you’re using currently, or are considering changing what you’re doing, read on for an overview of these different optimization approaches and their relative strengths and weaknesses.

Model-Based Optimization

Global keyword level: Every keyword receives individual analysis and an individual bid so the entire portfolio achieves the stated goal. The approach is difficult to accomplish because the size and complexity of many paid-search programs requires literally millions of pricing decisions to be made each day. But a solution that does accomplish global keyword-level modeling is likely to be pure software and, therefore, fully automated.

Local keyword level: In effect, this is the approach advocated by Hal Varian, Google’s chief economist: bid each keyword separately based on the predicted value. Simplicity is the key because you don’t need to predict behavior across a range of bids – just bid a percent of the predicted value. However, with simplicity comes lower performance and limited settings. In a lot of cases, a local solution leaves money on the table. You can set a target but you can’t layer on multiple constraints.

Global cluster level: Here you still have a global optimization but models based on clusters of keywords are used to handle the sparse-data problem. Some vendors are actually a hybrid of this plus the keyword-level global approach — that is, they use keyword-level for head terms and clusters for the tail. Global cluster-level bidding tends to be stable with results that are repeatable. But what you gain in stability, you lose in performance and automation. Performance suffers because clusters ignore the fact that every keyword is unique and the value of the aggregated data is outweighed by the loss of that uniqueness. A variety of factors such as seasonal changes, expanded keyword lists and changes in product offerings can render clusters obsolete. When obsolescence occurs, statisticians are typically needed to manually tune the models, so cluster-based solutions are rarely pure software applications.

Rules-Based Optimization

The most common solution available, rules-based optimization is touted for being simple and easy to understand. For example, a rule might state, “If ROAS is less than 200 percent, lower bids by 10 percent.” However, when rules are layered upon rules, simplicity and understandability are quickly negated, making it difficult to understand what will happen to the bids. The big loser in rules-based optimization is performance. Rules-based systems are reactive, with pre-defined responses to certain situations. In a rules-based system, historical data is not considered because the situation drives the reaction. Because of their reactive nature, rules-based optimization can be very good at protecting your position, but playing defense rarely leads to optimal results.

For a more granular look at these approaches, download OptiMine’s white paper (no registration required) “Achieving the Gold Standard in Paid-Search Bid Optimization”.

@markpalony

Tweet

In the ever-evolving world of paid search, Alex Cohen of Search Engine Watch has been keeping an eye on keyword-free PPC. In this non-traditional approach – which is especially relevant to e-commerce sites – your keywords in are replaced by attributes fed from your product catalog and ads are delivered as product search and product listing ads or text ads with product extensions. Cohen’s post breaks down the new keyword-free world, adding clarity with the use of several great examples.

Tweet

As a great soccer player’s wife once said, “So tell me what you want, what you really really want.”  She wasn’t talking about paid search bid optimization but marketers should consider, do you want impressions, clicks, leads, sales, revenue, or profit?  For some advertisers, any of those will do because their costs per impression, click, lead, sale, unit of revenue, and unit of profit are about the same regardless of their ad spend.  That is, each dollar of advertising generates about the same number of impressions, clicks, leads, sales, revenue, and profit regardless of how many dollars they spend (when properly optimized with an application like OptiMine’s).  For many advertisers though, this isn’t the case and they must pick what they really really want.  Here’s a case in point from retail financial services.

For this advertiser, keywords, and competitors, cost per lead is about $33 regardless of how much they spend.  Each incremental $33 of spend gets another lead.  However cost per sale is about $150 when spending $20,000 a day but $300 when spending $40,000 a day.  To understand this, consider a real example from the credit card business a few years ago, before the credit crunch.  Back then, if you bid on the paid search phrase “credit cards for people with bad credit,” then you would get a lot of applications (leads).  However, you got few sales (new accounts) since the people who search on that phrase rarely had their application approved, thus driving up the cost per sale.

In situations like this, if paid search marketing is managed to, say, a $35 cost per lead then they can generate thousands of leads per day.  But, on the other hand, if they’re managing to a $200 cost per sale then they’re limited to about 130 sales a day in paid search.  So, decide what you really really want, leads or sales, and at what price.

With automated bid optimization applications like OptiMine’s, businesses are increasingly using marketing metrics beyond cost per lead or cost per sale, including weighted sums of multiple metrics, customer equity, revenue return-on-ad-spend (ROAS), and profit ROAS.  There’s no one best metric for optimizing paid search marketing for all businesses, or even for all businesses in an industry sector like retail, retail apparel, or retail shoes.  It depends on strategy.  For example, revenue ROAS is customer-centric while profit ROAS might not be; one may be better aligned with your business goals in July while the other’s better in November.  It depends what you really really want.

Doug Bryan
Co-founder
OptiMine
www.OptiMine.com

_____________

Further reading:

1. Customer Equity, R. Blattberg et al., Harvard Business School Press, 2001
2. “Enhancing Customer Equity Through Add-On Selling,” R. Blattberg et al., Harvard Business School Press, chapter, 2001
3. Marketing Metrics, P. Farris et al., Wharton School Publishing, 2006

Tweet