When it just works
Pluspunten
It really saves my time as a web scraping specialists. I always need rotated proxies for my client needs. I keep creating a new project and it is a lot of hassle of I have to buy proxies, renew proxies, and create proxy rotation on my own. I am glad I don't even need to think about it. It just works!
Minpunten
Well, it's just doesn't have enough advanced documentation on Scrapy Splash combined with Crawlera. Their development team in Github is very responsive though.
Relatively Easy To Use
Our company, whatoplay.com, aggregates data from multiple sources around the world. Since we started using Crawlera, we haven't really encountered big issues. Our common ones in the last 6 months is just reaching our limit, so we just had to increase our plan.
Pluspunten
We're only using Crawlera for our data scraping needs and it has worked seamlessly since we started using it in 2016. When we had to upgrade to a higher limit, the transition was fast, and no dev time was wasted updating our existing codebase.
Minpunten
Pricing can be a bit tricky especially if your need is right in between the plans.
K-LIX
I have a good experience with scrappinhumb, in a matter of minutes I can develop and publish a new spider.
Pluspunten
The software delivers the exactly promised result. It has a friendly interface and an efficient support.
Minpunten
You need a roadmap of new features and the pricing of third-party services.
Antwoord van Scrapinghub
Thank you for the review. All new features will be announced in our Support Center.(https://support.scrapinghub.com/support/home) We also have an Ideas Forum (https://support.scrapinghub.com/support/discussions/forums/22000200101) where we would love your input on ideas or new features you would like to see. We appreciate all input from our Customers to continue to help us improve our products.
Why do you need that title?
I use and recommend that platform for years for my customers which need production-ready enterprise-grade data scraping systems.
Pluspunten
- Original and flexible technology
- No vendor-lock
- Easy to use for professionals
- Pretty convenient API system for integration with third-party solutions
Minpunten
- Not so easy to use by non-professional IT person that still wants to use data scraping
- Lack of ability to create some kind of simple and clear user interface for such a persons
- No simple solution for distributed/high-volume crawling
- Lack of monitoring and alerting, non convenient logging system
- Overpriced Crawlera
Antwoord van Scrapinghub
Thank you for your review we appreciate all customer feedback. We do have a managed service offering for non IT professional who want access to the Data without having any scraping skills.
Simple, reliable and powerful service
Never had a problem, and customer services always replied to our requests promptly, and with attention to detail.
Pluspunten
We are very happy with Crawlera. It works like a charm with our Scrapy projects, just adding a few lines of code we completely forgot about proxies that ended up failing anyway.
Minpunten
Some requests end up being very slow, but good news is most of them end up going through.
Fantastic If You Know How To Use It
Portia aside, I highly recommend this platform. It's cheap and works really well. If you know how to use Scrapy, this is easier than launching a bunch of VMs, etc. and relatively inexpensive.
I knock Portia because I wanted it to work, and it will with A LOT of patience, but, if you know your way around Python and Scrapy, this is the scraping cloud platform for you!
Again, there technical support has been great for me. Most of my issues have been stupid mistakes by me, but they are happy to help in a reasonable time frame (usually under 24 hours/ under 8 hours in my experience).
Pluspunten
- Works perfectly as long as you do not need Portia
- Fast
- Many available features in one place
- It fulfills my scraping needs well
- EXCELLENT technical support
Minpunten
- Portia (beta) is almost unusable. I got a couple very basic programs out of it before I taught myself how to use Scrapy. Still, if you are looking for a value and have time to deal with the lag/hiccups in Portia, it is by far the best value for a simple point and click scrapying platform that I found. You can make it work, you will just want to put your head through the screen before you are done...
Overall 10
Started using it 2 years ago, learning curve totally rely on the framework because the cloud platform is pretty intuitive, self explanatory and easy to use.
Whenever I needed support I have been response fast and with a favorable solution.
Pluspunten
A full integrated platform for a framework well done for it purpose. Easy to use, based on others frameworks structure, so if you are used to web development in python then it's a piece of cake to create spiders.
The integration (scrapy + scrapinghub) its really good, from a simple deployment through a library or a docker makes it suitable for any need.
Good support and constant improvement on the platform.
A lot of plugins and, open to any feature needed.
Minpunten
So far there is nothing I dislike about it.
scrapy review
Scrapinghub has enabled us to streamline the scraping process for our company and sets the foundation for future scaling. I like the fact that scrapyhub handles the "Ops" part of scraping like provisioning cloud resources, staging data, and rotating IP addresses.
Pluspunten
Scrapy is amazing, python api is easy to use, job dashboard is easy to understand, good documentation and code.
Minpunten
Dynamic pages sometimes hard to scrape, nothing wrong with lua (for splash) as a language, but can be minor barrier to entry for python/javascript users.
I love Crawlera
I am very happy with Scrapinghub. Whenever I need to run a small scraper that I can't run on my laptop (since I don't leave it turned on 24/7), I just run it on Scrapy Cloud. Meanwhile, Crawlera is just the best!
Pluspunten
I have been using Scrapy and Scrapinghub's services since 2013 and I'm so far very satisfied with their services. Crawlera, their proxy service, works very well! I don't have to setup a proxy farm anymore or configure my scrapers to point to thousands of proxy services as they do all the grunt work for you (it's all automated).
Minpunten
Minor hiccups from time to time on the dashboard. It only affects me when I want to see historical stats but this problem doesn't affect functionality.
Scrapinghub allows me to do things other proxy servers don't do.
Scrapinghub gives my business higher quality raw data so my clients get better results.
Pluspunten
This software allows me to access 97% of the web resources I need. Other proxy servers I have used gave me 75% maximum.
Minpunten
That its a time based subscription. I would prefer being able to buy a certain number of requests and renew when those run out.
Antwoord van Scrapinghub
Thanks you for your review and input on our pricing model - we appreciate all customer feedback and have fed this into our product team for review.
Best for scraping
The best out there for any kind of scraping need. They new AI based scraping api is a game changer
Pluspunten
The best out there for any kind of scraping need. They new AI based scraping api is a game changer
Minpunten
Nothing bad about it. Beats the competition out of the water
Amazing crawling solution
Coming from a "legacy" environment where everything was build from scratch I have to say that my experience with Scrapinghub is really positive. I started using some more advanced features such as ItemLoaders, Middleware etc.
Pluspunten
Scrapinghub is so easy to use. With few setups you're ready to build your first spider. The integration with github and other addons (such as crawlera) makes things even easier to manage code deploy and proxy network management. Support for Python 3 is also a great improvement
Minpunten
I'm still in early days with scrapinghub and at the moment did not see any big issues to take into account. One things that comes in my mind to be improved is the documentation.
Antwoord van Scrapinghub
Thank you for your review we appreciate all customer feedback and are constantly looking to have better documentation with continuous improvements underway. If you see any issues in our current documentation we would love to hear from you. Please submit via our Support Center.(https://support.scrapinghub.com/support/home)
good for scraping
Pluspunten
it helps a lot to avoid the self-hosted spiders, this is the primary reason I like it.
and I know it is developing to make it more ease to use, like github integration, however I have set up the CI tools to publish the spider automatically for every code change, so I don't use it.
Minpunten
however it is always a pain that it is more difficult to bypass the bot detection, especially for big website, like amazon, and the other websites protected by Distil. I know this is not the fault of scrapinghub, but it is a big pain. and I know you have a service Crawlera for this purpose, but it is quite expensive.
Antwoord van Scrapinghub
Thank you for your review we appreciate all customer feedback. We agree and antibot research is something we are also focused on to help our customers.
Great proxy and scraping utils with superb support
Pluspunten
I use the crawlera proxy system from Scarpinghub. It is very straightforward to use and results are great.
Minpunten
No cons until now. Once in a while a server/IP gets banned but they have a big pool of IPs to serve you.
Scrapinghub
Pluspunten
I use the Python scrapy module to write crawlers to monitor competitors prices. The integration with Github makes it real easy to deploy code to the hub. The dashboards are very useful to monitor progress and schedule jobs. Really happy with this offering.
Minpunten
In the beginning it was somewhat cumbersome to automate data extraction. Once I started using Python scrapinghub module life became easy. So also a pro!
Shub Review
From what I've used (basic spiders) it has been great, the page is really easy to use and the cli makes deploying easy.
Pluspunten
Open source, well documented and very efficient.
Minpunten
Splash needs to be migrated to another tool IMO. I don't find LUA easy to work with as a developer; not because of the language but debugging and maintaining code is really hard. If you use scrapy-splash there is a black box that cannot interact with. I think going mainstream with javascript or python whilst unifying browser scrapping and scrapy itself would be an amazing decision.
So, overall I would love to see a merge between scrapy and a browser scraper.
The best crawler service plataform
Crawlera is a great platform for crawler. I used it for 2 years in my business and jobs.
Pluspunten
Your reliability an trust about proxy ip list.
Minpunten
You payment method just accept international credit cards. I used it from Brazil.
Just Eat Client
Good, they are responsive and professional. Would recommend.
Pluspunten
As a business once you hit a certain size it makes sense outsourcing webscraping. Scrapinghub has the required specialism to do a good job.
Minpunten
Still needs involved work from the client side to ensure data quality and consistency.
Antwoord van Scrapinghub
Thank you for your review we appreciate all customer feedback. We agree and data quality is very important to us as well. We are constantly looking to improving our QA framework and working with our customers to ensure consistent data quality.
The best way to host your scrapers
Pluspunten
Very easy to use. Great dashboards for monitoring jobs. Very competitive pricing. Friendly customer service.
Minpunten
There's very little to complain about with Scrapinghub - it's genuinely a really great product!
The proxy is smart so I don't have to be!
Pluspunten
Crawlera makes it really easy for me to process thousands of requests without worrying about rate-limits. I love that I don't have to worry about retry policies or IP blacklisting because Crawlera takes care of that for me! Plus, by charging by requests instead of bandwidth, it's much easier for me to estimate my expenses based on my needs.
Minpunten
I don't like that the usage numbers are delayed by so many hours.
The pricing is also a little high. I wish there was some way to bring that down or get more value into the plans.
Highly recommended!
Pluspunten
The effectiveness - it simply works better and is more consistent than other proxies.
Minpunten
The monthly limit on requests and how there is no intermediary plan between C10 and C50.
Scrapinghub has revolutionized the scraping industry
I came to scrapinghub as a young ambitious analyst. The free to use service is what kept me. When more scraping projects comissioned I contacted other providers to find out who would deliver the most value. There really was no alternative: the best prices, virtually unlimited data, and A+ team at scrapinghub.
Pluspunten
Simplicity, no-bullsh*t services, modular services & addons, very low service downtime. I can keep going and still won't do these guys justice
Minpunten
Payment platform wasn't a walk in the park. Guides and knowledge base are not outdated but can use a 2019 refresh
Great for Beginners
I am solving meta search price problems within the cannabis industry.
Pluspunten
It was very easy to start creating crawlers and launching them on scrapinghub. Plug and play. Ready to start launching a fleet of crawlers to move my project to the next level.
Minpunten
The price was a bit hefty for running multiple crawlers.
Scrapinghub Review
Pluspunten
· Scraping situation is easy to see visually
· Can be finely implemented such as time to activate scraping
· Ease of deployment
Minpunten
· I try to deploy from github, but I can select only my own repository (Can not deploy from organization's repository?
eases web scrapping experience
would recommend anyone who web scraps website data for mining intelligence.
Pluspunten
we have used scrapinghub for generating proxy ip's to mine company web data and career page information. has been very useful in assisting us to stand up our own skill vertical search engine.
Minpunten
needs a lot of customization if the scripts run into issues. overall a good product for its use cases
It makes me work really fast but still secure for my clients. I look really professional even though what I did was only suggesting and installing Crawlera. It's awesome!