Self-hosted webscraper.
AnyCrawl 🚀: A Node.js/TypeScript crawler that turns websites into LLM-ready data and extracts structured SERP results from Google/Bing/Baidu/etc. Native multi-threading for bulk processing.
Web Scraper in Go, similar to BeautifulSoup
Scalable Python web scraping scripts for +40 popular domains
Command line tool to download and extract data from HTML/XML pages or JSON-APIs, using CSS, XPath 3.0, XQuery 3.0, JSONiq or pattern matching. It can also create new or transformed XML/HTML/JSON documents.
a class that uses scraped proxies to make http GET/POST requests (Python requests)
GoScrapy: High perfomance webscraping framework in Go, inspired by Python's Scrapy.
An R web crawler and scraper
AniWorld Downloader is a free, open-source app for finding and downloading anime, movies, series, and manga, with video streaming support. Available on Windows, macOS, and Linux, with a Web UI, terminal menu, CLI, and Docker support. If you enjoy using it, feel free to leave a ⭐!
An AI assistant tool that integrates coding, writing, and reading functions. For better alternatives see https://monica.im/desktop
📚 This is an adapted version of Jina AI's Reader for local deployment using Docker. Convert any URL to an LLM-friendly input with a simple prefix http://127.0.0.1:3000/https://website-to-scrape.com/
Self-hosted feed generation toolkit for turning webpages, email folders, APIs, and messy web sources into reliable RSS feeds.
Python class to scrape data from rightmove.co.uk and return listings in a pandas DataFrame object
WebExtractor is a powerful OSINT and ethical hacking tool developed in Python. It is used to extract email addresses, phone numbers, and links from a target website
Lego AI Parser is an open-source application that uses OpenAI to parse visible text of HTML elements.
A Python command-line tool for scraping and downloading subtitles from AppleTV and iTunes movie pages.
A web scraper to scrape email's and phone numbers from Websites.
DotnetCrawler is a straightforward, lightweight web crawling/scrapying library for Entity Framework Core output based on dotnet core. This library designed like other strong crawler libraries like WebMagic and Scrapy but for enabling extandable your custom requirements. Medium link : https://medium.com/@mehmetozkaya/creating-custom-web-crawler-w…
A letterboxd webscraper
Unofficial Python client for xvideos.com: search, metadata extraction, and video download (curl-cffi + selectolax)
To associate your repository with the webscraper topic, visit your repo's landing page and select "manage topics."