Splash authentication credentials potentially leaked to target websites in scrapy-splash

CVE-2021-41124

What is CVE-2021-41124?

Scrapy-splash is a library which provides Scrapy and JavaScript integration. In affected versions users who use HttpAuthMiddleware (i.e. the http_user and http_pass spider attributes) for Splash authentication will have any non-Splash request expose your credentials to the request target. This includes robots.txt requests sent by Scrapy when the ROBOTSTXT_OBEY setting is set to True. Upgrade to scrapy-splash 0.8.0 and use the new SPLASH_USER and SPLASH_PASS settings instead to set your Splash authentication credentials safely. If you cannot upgrade, set your Splash request credentials on a per-request basis, using the splash_headers request parameter, instead of defining them globally using the HttpAuthMiddleware. Alternatively, make sure all your requests go through Splash. That includes disabling the robots.txt middleware.

References