Doing Keyword Research The Smart Way in 2020

It’s every content site owners dream to find every post you publish in the top 3 (if not top 1) of the SERPs. Perhaps you’re no stranger to this goal, so you have a solid SEO game plan to boost your visibility… or so you thought.

Green light from a keywords plugin? Check. Complete switch to HTTPS? Check. Loading time in under 2 seconds? Check. You were so confident that you followed all the best practices of SEO, but why is it still not working? Even your constant stream of daily 2,000 word articles did nothing to stop your website from losing visitors.

Whether you’re a beginner niche site builder or a veteran managing authority sites, it pays to learn about the core foundation of SEO: keyword research. Count on this guide to give you a complete walk through on how to find, monitor, and optimize keywords for your posts.

In the later part of the article we will cover some of the nitty-gritty of what works to find low competition keywords in any niche.

1.What is a keyword?

Basically, a keyword is the totality of all terms entered by a user when he uses a search engine. If you typed in “how to find and analyze keywords” before finding this article, that entire phrase counts as a keyword. It also goes by the names key phrase, search word, or search term.

The exact keyword used by readers are hard to predict, for it often varies according to a lot of behavioral factors. Take it from Google who revealed that 15% of all queries they process actually include terms that no one has ever searched before.

A quick heads up: in this guide, we’d only be exploring keyword research and analysis for Google. With a global market share of 90%, it has by far the most studied algorithm for creating SEO tactics. There are other search engines like Pinterest, Bing, DuckDuckGo, Amazon, YouTube, Yandex, Baidu or Ecosia, but maybe we’ll have time for that in another article.

2. What is keyword research?

As an expert, blogger, or entrepreneur, you’re supposed to be a natural problem solver. To be successful in SEO, you have to find out what problems user seek to solve in the first place. This is where keyword research or analysis makes a red carpet entrance. It is simply the process through which you find out what keywords potential customers often enter into Google.

Keyword research has three main purposes:

• Find out why users are using Google. Are they looking for solutions o an issue? Or are they after tips to help achieve their goals?
• Determine what exact keywords are often used by your target readers.
• Analyze if using the highly searched keywords are worth integrating to your content. Is it reasonable to join the competition? Does the keyword imply interest to buy?

3. Why is keyword research important?

While other SEO tactics (e.g. link building and load time optimization) ensure that you appear in the search results, keyword analysis gives you control over what type of searches it would appear for. Sure, you may get a lot of visitors even if you skipped this. However, you end up settling with the bare minimum of your visitor count potential.

Trust me. After sitting down for a few days just to optimize keywords for 10 published articles, I’ve seen many authority sites enjoy up to ten times higher traffic from Google.

If that reason isn’t amazing enough, here’s a few more bonuses to convince you to take keyword research seriously:

• They make great writing prompts for your editorial plan, content schedule, and even market research.
• It allows you to understand your customers better. Keyword research lets you know their wishes, fears, problems, and even vocabulary level.
• It can give you a forecast on whether or not an article will be well received or not.

Let me share my personal success story with keyword research.

After letting an article accumulate dust for so long, I decided to finally optimize it. New search terms. Updated keyword cluster. I made sure all relevant keywords were seamlessly integrated. It probably took me no more than 30 minutes to put everything into place.

Before I gave it much attention, that post gets under 50 clicks per day. Guess what – simple optimization increased that record by 50%! This case actually involved one of our revenue generating articles. So apart from higher visitor count, my website also enjoyed $435 higher earnings in a month. Not bad for half an hour’s work, huh?

4. What tools can help me with keyword research?

Consider this checklist of tools to get you started with keyword research:

• Google Suggest – Simply scroll down to the bottom of a Google results page to find the related searches suggestions – a gold mine of keywords. You can also do the same trick on YouTube, Amazon, Pinterest, and Bing.
• Keyword tool – This will help you back your research with quantitative data. Superb apps will let you know a keyword’s search volume, strength of competition, and cost per click. I personally use KWFinder with a basic version costing you around $330 a year. If you’re on a tight budget, you can check out a free alternative called Ubersuggest. As I needed more advanced analytics, I also sought help SiteProfiler for competitor analysis, and SERPWatcher for keyword monitoring.
• Suggest tools – Tools like Answer The Public will help you find long-tail keywords and users’ search intent
• Thesaurus – This helps you find synonyms for keywords, so you could further expand your term bank (e.g. OpenThesaurus or Woxikon)

Expert tip: Google Ads Keyword Planner is also a popular choice among keyword tools. However, I find it not worth the hassle. You can use it to build your list of terms, but you’d have to use other tools for if you need more advanced and essential analytics.

5. What are the most important keyword properties?

Keywords are not as simple as you may think they are. While a quick brainstorming activity helps in making a rough list, there are several factors that can make or break your SEO strategy. A rushed keyword research may give you more visitors, but not necessarily more customers.

To optimize your choice of keywords, let’s explore its most important properties that you should consider:

5.1 Word count

The number of words in a key phrase determines how specific it is, and they are categorized as either:

• short-tail (e.g. shoes) – Often consists of a single word, these keywords have the highest search volume. However, competition is harsher and conversion rates can be quite low.
• long-tail (e.g. Nike running shoes for women) – In contrast, these keywords are more detailed. There would be less competition and increased conversion rates.

Don’t get turned off by short-tail terms so easily. Here’s a tip: general terms work best as focus keywords. This means that they can give you better results if they are repeated in the most important parts of your article (e.g. title, meta title, and meta description).

5.2 Search volume

Basically, search volume indicates how many users have entered a keyword on Google each month. General rule of the thumb says that the higher a keyword’s search volume is, the more visitors it can lead to your website.

Make sure your chosen keyword tool can display this property! My personal favourite is KWFinder, which helps me identify and get rid of terms that have less than 500 queries per month.

Expert tip: Optimize your content not based on a single keyword, but on a keyword cluster. This is basically a group of search terms related to a single theme. You’ll know more about this on the steps on conducting a keyword research.

5.3 Competition

Finally ranking first on Google search results should not make you rest so easy. Reality check – this doesn’t really guarantee that ALL users will visit your page! Most traffic is distributed among the top four results, with you getting only a 20% share from the cake. If you’re found deep in the second page (rank 11 to 20), it’s as if you’re not in the run at all.

Not wonder this joke is so popular among SEO geeks:

Where is the best place to hide a body? On the second page at Google!

Click rates can be affected by many factors too, so don’t stop optimizing. Check your result snippet and see if everything looks perfect. Maybe your brand isn’t that popular so people just skip it. SERP features (e.g. Google shopping ads) can also affect which result is most appealing.

This leads us to another reason why I love KWFinder so much. It makes my life so much easier with an integrated competition analysis! From various back link metrics, the Landing Page Strength (LPS) for each keyword is determined. Using LPS, the tool will then calculate the keyword SEO Difficulty on a scale of 0 to 100. This gives you an insight on how difficult it is to get on the first page.

5.4 Search Engine Results Page (SERP) features

Depending on the search term and user, Google displays SERP features along with normal or organic search results. These include:

• Text ads via Google AdWords (e.g. products or services with ratings, call-to-action buttons, phone numbers, and variants)
• Notes on matching mobile applications
• Google Maps
• Google Shopping
• Google News
• Related pictures
• YouTube-Videos
• Direct Answers or Google Knowledge Graph – a box where the search query is answered directly by Google
• Featured Snippets – text excerpts from a website that directly answers a search query

These SERP features could mean bad news for you, because these attention grabbing elements take more click rates away from organic search results like your article.

Let’s assume that you’re into the culinary arts niche, with an article on the best types of oil to use in the kitchen. When a user is lazy enough to just type “coconut oil,” a frustrating amount of SERP features will take most of the space in the search results. Two ads on the top. The Google Knowledge Graph on the right. It doesn’t end there! Google shopping results and news headlines will push the first rank organic result further down, to the point that its click rate will be less than 15%.

As disastrous as this may sound, the abundance of SERP features for a keyword actually indicates is profitability. The fact the brands are willing to pay money just to be bumped up its results page is a huge green flag.

5.5 Cost per click (CPC)

In relation with SERP features, CPC shows how much you have to pay for a click if ever you decide to place an ad for a specific keyword. The equation is simple: the higher the CPC, the more advertisers you have to compete with. Joining these ad wars can be worth it if you’re selling your products from your own business or via affiliate marketing.

Wanna know how serious the battle for an ad spot gets? For a single click, advertisers are eager to pay as much as $49 for the best keywords in car insurance. Emphasis on for just a single click!

5.6 Search intent

To give users the most relevant results, Google’s algorithm give much importance to “search intent.” Whenever you enter a query, their AI tries to make an intelligent guess on:

• The goals or problems you are trying to solve
• Your expected type of search results
• Other related topics that might spark your interest

Learn from Google itself and try to integrate search intent into your keyword research. By doing so, you make sure that the visitors you attract are those who you can generate revenue from. These could be people who are now ready to buy products and services, fond of clicking on affiliate links, and willing to subscribe to your e-books and newsletters in exchange for personal data.

Moreover, search intent can be your compass in deciding what type of content to publish. Should you go for a review article? Or perhaps a tutorial could work better. This way, you greatly reduce the chances that a visitor would barely spend time on your post just because they didn’t find what they were looking for.

• Transactional (Do) – A user searched a query because she wants to do something. To simply listen and dance to Beyonce’s song in the shower, she could type “Beyonce Love on Top Music Video”
• Commercial – An additional subcategory for “do,” the user is interested to make a purchase and wanted Google’s help to finalize that decision. Instead of just looking for a music video on YouTube, she could instead type “amazon beyonce love on top single”
• Informational (Know) – Users with this search intent are looking for specific information, or solution to a problem. According Google, these keywords make up about 50 to 80% of all search queries. To make the most out of this intent, focus on producing content that has terms like “how to,” “guide,” “why is,” or “tips.” Long-tail keywords work best too.
• Know commercial – An additional subcategory for “know,” keywords that users type have reference to a product or service. These can be a great market for authority site owners who tend to produce content with the terms “test,” “comparison,” or “review.”
• Know the news – An additional subcategory for “know,” users searched with the intent to look for the latest information on a topic. This is why Google usually displays headlines from Google News when triggered with the key phrases “syria war,” “COVID-19,” or “US elections.”
• Simply Know – Unlike the “know” intent which often needs in depth discussions, this classification are for users whose query can be answered quick – maybe with just a table, exact statistic, or a single word (e.g. opening hours IKEA, number of states US, etc.). Google rarely displays organic search results for these, so don’t bother to optimize your content for this type.
• Navigational (Website) – This involves a user wishing to go to a specific website by entering a brand name, domain, or even an entire URL. Actually, it can be another gold mine for niche sites with affiliate marketing. Using brand names and their products strategically can bump your post up the results page.
• Local (Personal Visit) –  With these search terms, users look for local places (e.g. restaurants, cafés, petrol stations, or ATMs) that they could go to in the future. These can be useful for increasing your audience in a specific region.

5.7 Phases of the customer journey

Another important question you have to ask is, “What phase of the customer journey is the searcher in?” Turning your back from this factor can compromise your conversion rates. Sorry sweetie, but you can’t be high on optimism and just assume everyone who visits your niche site will buy something.

The customer journey can be divided into these five phases:

• Inspiration – Awareness for the product is awakened
• Favorability – The interest in the product is increased
• Consideration – The customer is considering buying the product
• Impulse – The intention to purchase becomes concrete
• Conversion – The product is finally bought

This factor should be closely considered with a user’s search intent. Keywords classified into the relevant types of intention (e.g. Know, Commercial, Do Commercial, and Navigational) can generate higher conversion rates when backed up with a study on visitors’ customer journey.

5.8 Search Volume Trend

Quick facts on search volumes will only give you insights on its average per month. Take your analysis a step further and look at the trends too. In what months was the keyword most searched? Is its use gradually increasing? By asking these simple questions, you can better understand the external factors (e.g. time, social trend, holiday, etc.) which may affect your visitor count and revenue too.

5.9 Seasonality

To make sure you focus on the right content at the right time, look out for seasonal variations in your keywords. While it’s obvious when summer-themed articles will work best, there are terms whose seasonality can only be determined if you dive deep into its search volume and other surrounding contexts.

This is popular among current affairs related keywords. For example, queries about the General Data Protection Regulation (GDPR) sky rocketed between April to June, 2018, and websites were able to ride the wave by publishing content a month before the law was passed on May 25, 2018.

6. What are the steps in conducting a keyword research?

Keyword research is just as important as content production itself. Trust me, you wouldn’t want to disrupt your writing momentum just because you suddenly couldn’t think of the right long-tail term for your fourth subheading. I personally spend days for this process, and here are the steps I often go through:

6.1 Brainstorming

Straight A’s in all English subjects isn’t a requirement to be successful in keyword research. Wanna know the super power of SEO experts? Empathy. The better you understand the person behind each search query, the more impressive your keyword bank would be.

Let’s assume that you are a meditation coach who would love to share your insider knowledge and tips on your website. While brainstorming for keywords, the best questions to ask are:

• What problems and goals do your desired customers have with meditation? Probably, it’s torture to sit still for an entire 15 minute session. Or perhaps they want a chill way to manage their anger.
• What questions do interested customers usually ask before trying out meditation?
• Despite all the people physically around them, what caused them to ask Google? Are they embarrassed of their problem? Are they skeptical of non-expert comments?
• What type of solutions are they looking for? Steps on how to meditate? Reviews about yoga apparels and equipment?
• Can any of your offered services (e.g. online courses, webinars, meditation app, etc.) help them?

These basic questions, guaranteed, is enough to give you a long list of topics and keywords ideas to last you for the first months of your niche site.

Expert tip: For far greater results when brainstorming, try to create a “customer avatar.” Basically, vividly imagine an actual person as you think deeper about his problems, goals, personality, and interests.

6.2 Find keyword ideas

From all the abstract concepts you wrote down after your brainstorming session, you now have to collect keyword ideas. Don’t worry about categorizing or filtering them yet. We’ll come to that later. Just list anything and everything you could think of.

Here are some hacks you can use to finish this step in a breeze:

• Sneak into a search engine’s suggest functions – I’d love to call this hack as “brainstorming part two.” Start off by entering a keyword on the search bar. Notice how Google’s auto-complete function greets you with a list of queries? Jot ’em all on your notes. Click on any of those suggestions, and scroll all the way down to the related searches section which, for the nth time, is a legit gold mine of keywords. You can also check out SERP features displayed, especially Google’s Direct Answers. You could always take your research miles and miles farther by doing the same process with other search engines like Bing, Pinterest, or YouTube.
• Check what keywords your competitors are using – Put on your spy hat and get ready to scout the competition. Using Ubersuggest, enter the domain of the brand you want to check out. Deep below the result overview, you will find the end of the rainbow (and a huge pot of gold) shining on a section labeled “SEO keywords” This is a list of all the search terms for which that domain is raked. If that isn’t enough, simply click “show all SEO keywords” for more treasure.

6.3 Filtering and sorting results

Three steps down comes the time to trim down your pile. Start off by weeding out all the duplicates and thematically irrelevant search queries. Done? Now, launch your favorite keyword tool and get ready for more serious action.

I will never get tired of setting you up with KWFinder (but hey, no one’s stopping you from checking out other tools we have listed in section 4, okay?). I love how it immediately displays an overview for each term, including search volume, strength of competition, CPC, and sample search results. Basically, you have take note of all the keyword properties we’ve talked about earlier and get a vibe on whether or not a keyword is worth the effort or not.

After all that hard work, export your data (worry not, most keyword tools have easy peasy export functions) and group them into tables. I personally like to print them out and work with colored markers, and sometimes scribble a few notes in the margins too.

6.4 Finding a suitable main keyword

Within each group or category you’ve formed, you now have to select the best focus keyword among the pool of terms.

To get a clearer picture on how to do this, let’s go back to your meditation coach persona. You may have one group labeled “introduction.” From all of the words and phrases in that category, it’s no surprise that you were drawn to the query “meditation for beginners.” Besides, its high search volume and moderate competition makes it perfect to focus on.

6.5 Form a keyword cluster

Take all of your main keywords and prepare to design “keyword clusters.” Simply put, it’s through which you will extend your focus towards related secondary keywords classified into the following categories:

• Synonyms – playing with synonyms is a basic hack to make articles less boring and monotonous. You can also make sure that your term can be understood by users with varying vocabularies. Apart from the main keyword “beginners,” you can also use “dummies” or “newbies.” You can quickly find synonyms with tools like KWFinder, OpenThesaurus, or Woxikon.
• Long-tail keywords – For instance, “meditation for beginners” can be connected to the queries “easy yoga positions for newbies” or “things you need before your first meditation class.” Long-tail keyword ideas can be found in Google’s auto-complete suggestions, as well as tools like KWFinder, Ubersuggest, and Answer The Public.
• W questions – This is just a subcategory under long-tails. As the name suggests, these are questions of “when,” “where,” “why,” and “what” related to your main keyword. Optimizing your articles with W questions can make it more accessible through digital assistants like Alexa and Siri. Moreover, they can be used as great subheadings for your blogs. For your meditation for beginners article, you could explore the queries “when was meditation discovered,” or “when is the best time to do meditation.”

I know your brain cells are quite tired at this point, but don’t skip this step to reap the following benefits:

• Even if a lot of big guys already dominate your main keyword, you have a strong fighting chance to rank for the secondary search terms
• You’ll cover a lot of user intent in a single article, a great boost in ranking for the main keyword
• With synonyms and related terms, you won’t turn readers off with annoyingly repetitive keywords

Expert tip: Always write long articles! This way, you can cover the most number of secondary keywords and user intentions.

6.6 Keyword analysis after publication

Even after you write and publish articles, keyword research should not stop. Give your new post a few weeks, then check out how it was ranked and make appropriate adjustments. To do this, you can seek the help of Google Search Console’s search analysis function. Conveniently, it will give you quick insights on the visitor count from each search query, your rank itself, and your click rate.

7. Frequently asked questions

7.1 How much will keyword research cost me?

There are a lot of ways to work on a tight budget, or even with the total absence of it. You can try out free tools like Answer The Public and Ubersuggest, or do things manually through Google search. But if you want to save your precious time and effort, perhaps paid keyword research will be worth the investment. With a subscription price of about $100 per month, you can enjoy advanced analysis features of SEO tools like SEMRush and Ahrefs.

7.2 How long will keyword analysis take?

Finding, sorting, and grouping keyword ideas can take several days, depending on the size of your niche and target visitors. Meanwhile, I personally spend about 1-3 hours to select a main keyword and design clusters.

7.3 What is the best software for create keyword lists?

My personal favorites are Microsoft Excel (more suited for professionals and experts in advanced functions) and Google spreadsheet (works best for beginners and enables easier, clearer workflow). You can also try using Calc, the spreadsheet program from the LibreOffice suite.

7.4 How can I find out the search volume for several keywords at once?

Analyzing keywords one by one will give anyone a terrible headache. For mass retrieval of keyword data, you will need the paid tools KWFinder, SEMRush, or Ahrefs. If you’re still hesitant about spending, the free searchvolume.io allows you to find the search volume of up to 800 keywords at once.

7.5 Which keywords have the highest search volume?

Keywords with brand or website names often have the highest search volume, thanks to people who would rather enter that site’s name on Google rather than type its complete URL in the address bar.

Take a look at the top 5 most searched keywords in the USA from Ahrefs’ ranking list (as of July 2019):

• facebook (233,100,000 searches)
• youtube (195,600,000 searches)
• amazon (104,800,000 searches)
• gmail (92,530,000 searches)
• google (84,920,000 searches)

7.6 How do I best find user’s search intentions?

Apart from our tip on using empathy so you could understand visitors better, you can also enter a keyword on Google and create a cluster out of the top 10 organic search results.

7.7 Is there a recommended WordPress plugin for keyword analysis of a text?

To be honest, the blogging world have yet to develop a tool that I can consider as absolute god-sent. All available keyword analysis plugins for WordPress (e.g. Yoast SEO, Rank Math, and SEO Squirrly) are too focused on the main keyword and ignores secondary terms.