SEO or SEM Banner for web development

What do they stand for?

SEO = Search Engine Optimisation

SEM = Search Engine Marketing

*Another related term that will be heavily mentioned throughout this article is SERP (Search Engine Result Pages).

How does it work?

Before we learn the difference between SEO and SEM, let us dive into how Search Engine works in order to better understand things. The 3 main factors about Search Engines are:

  • Algorithms

    Search Engines such as Google have their own algorithms. Little is known about how the algorithms are calculated and Google has never publicly stated which algorithms it uses. Therefore, people are not able to beat the system by filling their site or pages irresponsibly to raise their ranking on the SERPs. The algorithms are regularly reviewed and refined to prevent any new methods that are used to cheat the system.

    To quote an example, at one point it was discovered that one of the factors of Google’s algorithms was to repeat keywords in a blog. This caused many bloggers to spam the keywords in white, in-between paragraph spacings. The keywords were detected by bots but not seen by the reader. This resulted in many blogs ranking high not because the contents were useful but because the keyword was repeatedly mentioned.


Photo taken from SiteChecker. Click here to read the list of updates in-depth.

  • Crawlers

    Knowing the algorithm would be useless without bots known as “Crawlers”. Crawlers are a type of program that visits every site and pages thoroughly. For example, Google’s crawlers are known as “Google Crawlers”. It records the entire page’s contents and saves them to the search engine provider’s storage. These crawlers not only record the site’s data but also scan through the contents and use the algorithms as checklists to rate the site.

    Crawlers are constantly evolving and repeatedly upgrading. A simple crawler that just visits and records sites has since been supported with features such as the ability to detect sites using white texts to raise SEO.

seo crawler web development

Picture taken from Search Engine Land.

  • Index & Indexing

    With the algorithms as a checklist and the crawlers as bots to explore and retrieve site data, you might ask what is index or indexing. Indexing is a process in which the data brought back by the crawlers are stored in a database. Index is the database where the recorded data are sorted into their ranking according to their SEO and other factors.

    In simple terms,

    • Indexing is the retrieving and recording of site data by crawlers.
    • Index is the records of the data collected and sorted.
seo crawler web development step

Image taken from The Next Scoop. Click here to read more about how indexing works in-depth.

  • Search Engine Result Page (SERPs)

    This is where the difference between SEO and SEM is much more obvious. The crawlers visit and rate sites based on their SEO-friendliness according to their engine’s algorithms. Depending on the rating, the sites are listed on SERPS according to their ranking. The results listed are based on