Intro :
All the websites that you are going to scrap either have dynamic or static data. The knowledge of the data type of the website will let you choose the right library and framework for web scraping.
Let's get familiar with both static and dynamic data.
static Data :
Static data refers to data that is present on a website but does not change or update dynamically. Examples of static data could include information about a company's products or services, contact information, or a list of news articles. Such data can be easily scraped using automated tools such as web crawlers or scraping libraries. In general, the website, in this case, does not use javascript rendering.
Dynamic Data :
Dynamic data refers to the content on a website that is constantly changing or updating, such as user-generated content, stock prices, or news articles. This type of data is typically generated by the server on the fly in response to user interactions or other events and is often stored in a database, as well as the data is not directly available in the HTML source code of the website.
What are the right tools for Static Data :
-> BeautifulSoup
-> Requests
-> Html5lib
What are the right tools for Dynamic Data :
-> Selenium
-> Scrapy