souvenir
Tag cloud
Picture wall
Daily
RSS Feed
  • RSS Feed
  • Daily Feed
Filters

Links per page

  • 20 links
  • 50 links
  • 100 links

Filters

Untagged links
8 results tagged scraper  ✕   ✕
ScrapeGraphAI https://github.com/ScrapeGraphAI/Scrapegraph-ai
10/12/2024 cluster icon
  • Scrapy : Scrapy is a fast high-level screen scraping and web crawling framework, used to crawl websites and extract structured data from their pages. It can be...
  • Upton : Upton is a framework for easy web-scraping with a useful debug mode that doesn't hammer your target's servers. It does the repetitive parts of writing...
  • Clint : Clint is a python module filled with a set of awesome tools for developing commandline applications.
  • mock : A Python Mocking and Patching Library for Testing
  • SpaCy.io : spaCy is a library for industrial-strength natural language processing in Python and Cython. It features state-of-the-art speed and accuracy, a concis...

ScrapeGraphAI is a web scraping python library that uses LLM and direct graph logic to create scraping pipelines for websites and local documents (XML, HTML, JSON, Markdown, etc.).

python scraper library llm
Colly http://go-colly.org/
24/05/2021 cluster icon
  • Goutte : Goutte is a screen scraping and web crawling library for PHP. Goutte provides a nice API to crawl websites and extract data from the HTML/XML response...
  • web.go : web.go is the simplest way to write web applications in the Go programming language. It's ideal for writing simple, performant backend web services.
  • Revel : A high-productivity web framework for the Go language.
  • Ponzu : Ponzu is a powerful and efficient open-source HTTP server framework and CMS. It provides automatic, free, and secure HTTP/2 over TLS (certificates obt...
  • Beego : An open source framework to build and develop your applications in the Go way
thumbnail

Colly is a Go framework that provides a clean interface to write any kind of crawler/scraper/spider

With Colly you can easily extract structured data from websites, which can be used for a wide range of applications, like data mining, data processing or archiving.

go scraper crawler framework datamining archive
Scrapoxy https://scrapoxy.io/
23/06/2020 cluster icon
  • Envoy Proxy : Envoy is an L7 proxy and communication bus designed for large modern service oriented architectures.
  • Portia : Portia is a tool that allows you to visually scrape websites without any programming knowledge required. With Portia you can annotate a web page to id...
  • Cool QL Cool : CoolQLCool (CQC) is an open source GraphQL server that allows you to turn websites into GraphQL APIs.
  • Rancher : Rancher is open source software that makes it simple for organizations to deploy a private container service and deliver Docker orchestration to users...
  • Orchestra toolbox : Orchestra is an open-source for live music performances in metaverse environments. Orchestra includes components for live performances in A-Frame/NAF-...
thumbnail

Scrapoxy hides your webscraper behind a cloud.

It starts a pool of proxies to relay your requests.

Now, you can crawl without thinking about blacklisting!

opensource scraper proxy
Cool QL Cool https://coolql.cool/
30/12/2018 cluster icon
  • Appwrite : Appwrite is an end-to-end backend server for Web, Mobile, Native, or Backend apps packaged as a set of Docker microservices. Appwrite abstracts the co...
  • Postal : Postal is a complete and fully featured mail server for use by websites & web servers. Think Sendgrid, Mailgun or Postmark but open source and ready f...
  • WireMock : Free and Open Source tool for building mock APIs. Create stable development environments, isolate yourself from flakey 3rd parties and simulate APIs t...
  • Matrix : Matrix is an open standard for decentralised communication, providing simple HTTP APIs and open source reference implementations for securely distribu...
  • GraphQL : GraphQL is a query language for APIs and a runtime for fulfilling those queries with your existing data. GraphQL provides a complete and understandabl...

CoolQLCool (CQC) is an open source GraphQL server that allows you to turn websites into GraphQL APIs.

graphql scraper opensource server api
Portia https://github.com/scrapinghub/portia
21/02/2016 cluster icon
  • Caire : Caire is a content aware image resize library
  • mob : Smooth git handover for remote pair/mob programming. mob is an open source command line tool written in go mob is the fastest way to hand over code vi...
  • Danswer : Danswer is the open source AI assistant connected to your company's docs, apps, and people.
  • Roots : Roots helps you build better WordPress sites faster. Open-source tools for WordPress application development. LEMP stack Project boilerplate Starter ...
  • Petri : Perti is Wix’s holistic experiment system, it covers the lifecycle of product experimenting including A/B tests and feature toggles.
thumbnail

Portia is a tool that allows you to visually scrape websites without any programming knowledge required. With Portia you can annotate a web page to identify the data you wish to extract, and Portia will understand based on these annotations how to scrape data from similar pages.

scraper opensource tool
Upton https://github.com/propublica/upton
04/02/2014 cluster icon
  • Scrapy : Scrapy is a fast high-level screen scraping and web crawling framework, used to crawl websites and extract structured data from their pages. It can be...
  • ScrapeGraphAI : ScrapeGraphAI is a web scraping python library that uses LLM and direct graph logic to create scraping pipelines for websites and local documents (XML...
  • Zero Clipboard : The Zero Clipboard library provides an easy way to copy text to the clipboard using an invisible Adobe Flash movie, and a JavaScript interface. The "Z...
  • vegeta : Vegeta is a versatile HTTP load testing tool built out of a need to drill HTTP services with a constant request rate. It can be used both as a command...
  • DD_belatedPNG : better PNG background-image support in IE6
thumbnail

Upton is a framework for easy web-scraping with a useful debug mode that doesn't hammer your target's servers. It does the repetitive parts of writing scrapers, so you only have to write the unique parts for each site.

scraper ruby library
Scrapy http://scrapy.org
27/09/2013 cluster icon
  • ScrapeGraphAI : ScrapeGraphAI is a web scraping python library that uses LLM and direct graph logic to create scraping pipelines for websites and local documents (XML...
  • Essentia : Open-source library and tools for audio and music analysis, description and synthesis
  • Goutte : Goutte is a screen scraping and web crawling library for PHP. Goutte provides a nice API to crawl websites and extract data from the HTML/XML response...
  • pyo : pyo is a Python module written in C to help digital signal processing script creation.
  • notifiers : A Python one stop shop for all notification providers with a unified and simple interface.

Scrapy is a fast high-level screen scraping and web crawling framework, used to crawl websites and extract structured data from their pages. It can be used for a wide range of purposes, from data mining to monitoring and automated testing.

python scraper crawler library
Goutte https://github.com/fabpot/Goutte
23/11/2011 cluster icon
  • Mink : Mink is an open source acceptance test framework for web applications, written in PHP 5.3.
  • PHP Quick Profiler : A tool for quickly profiling variables and resources used by PHP.
  • GrumPHP : Sick and tired of defending code quality over and over again? GrumPHP will do it for you! This composer plugin will register some git hooks in your pa...
  • Chromeless : Chrome automation made simple. Runs locally or headless on AWS Lambda. Chromeless can be used to... Run 1000s of browser integration tests in paralle...
  • Colly : Colly is a Go framework that provides a clean interface to write any kind of crawler/scraper/spider With Colly you can easily extract structured data ...
thumbnail

Goutte is a screen scraping and web crawling library for PHP.

Goutte provides a nice API to crawl websites and extract data from the HTML/XML responses.

php crawler test scraper
1634 links
Shaarli - The personal, minimalist, super-fast, database free, bookmarking service by the Shaarli community - Theme by kalvn