Colly is a Go framework that provides a clean interface to write any kind of crawler/scraper/spider

With Colly you can easily extract structured data from websites, which can be used for a wide range of applications, like data mining, data processing or archiving.

Chrome automation made simple. Runs locally or headless on AWS Lambda.

Chromeless can be used to...

• Run 1000s of browser integration tests in parallel ⚡️
  Crawl the web & automate screenshots
• Write bots that require a real browser
• Do pretty much everything you've used PhantomJS, NightmareJS or Selenium for before

Scrapy is a fast high-level screen scraping and web crawling framework, used to crawl websites and extract structured data from their pages. It can be used for a wide range of purposes, from data mining to monitoring and automated testing.

Goutte is a screen scraping and web crawling library for PHP.

Goutte provides a nice API to crawl websites and extract data from the HTML/XML responses.