- It allows the user to type in a specific product name, to crawl multiple e-commerce sites and asynchronously return useful results to the user.
- UI and ArchitectureBuilding the front endTo make it short, the front end is really small (only two pages: one for the search box and another one for the results).
- I moved everything from ‘search term’ to ‘search results items’ into Redux’s state, thus making the code structure cleaner and nicer.
- Some other problematic factors include: – Not having pre-stored information that could be useful to make predictions.Each result item consisting only of the following data: title, price, URL and thumbnail.My conclusion was to use a weighted ranking algorithm.
- In a more complete implementation, that data could be useful for some sort of Machine Learning algorithm to predict the most useful way to rank results.
This is a quick story about how I created a simple but end-to-end e-commerce scraping web application called “crawl-io”. The goal behind this article is to provide an example of an implementation so…
Continue reading “Building an eCommerce Scraper with Node.js and React.js”