NotoriousArnav

Web Scraping as a Service! [CrawlX]

Author: NotoriousArnav

Published on: June 4, 2024, 7:21 p.m.


So, I made a API called CrawlX which helps you do Web Scraping. Let me show you what I mean!<br />  

<u>Request</u>:

 

curl -X 'POST' \
  'https://crawlx.onrender.com/extract' \
  -H 'accept: application/json' \
  -H 'Content-Type: application/json' \
  -d '{
  "url": "https://results.eci.gov.in/PcResultGenJune2024/index.htm",
  "selectors": {
     "table": {
        "css": "tbody tr",
        "multiple": true,
        "type": "Text",
        "children": {
           "name": {
              "css": "td:nth-child(1)",
              "type": "Text"
            },
           "won": {
              "css": "td:nth-child(2)",
              "type": "Text"
            },
           "landing": {
              "css": "td:nth-child(3)",
              "type": "Text"
            },
           "total": {
              "css": "td:nth-child(4)",
              "type": "Text"
            }
        }
      }
  }}'

 

<br /> <u>Response</u>:

{
  "data": {
    "table": [
      {
        "landing": "3",
        "name": "Bharatiya Janata Party - BJP",
        "total": "241",
        "won": "238"
      },
      {
        "landing": "4",
        "name": "Indian National Congress - INC",
        "total": "99",
        "won": "95"
      },
...
      {
        "landing": "0",
        "name": "All India Majlis-E-Ittehadul Muslimeen - AIMIM",
        "total": "1",
        "won": "1"
      },
      {
        "landing": "0",
        "name": "Independent - IND",
        "total": "7",
        "won": "7"
      }
    ]
  }
}

So as you can see here, no need for client side libraries except the http request maker like axios. This is especially useful, if you are creating Single page applications using React/Vue or etc, which will essentially turn any SSR website into a API, if they don't already have one!

 

I have created an Example one with this!

Project: https://notoriousarnav.github.io/simple_alpine_axios_prj/

 

So please do try out my API!

API Docs

API Base URL

Comments
No Comments Found
More from NotoriousArnav
More Like This