Deviantart (or other art community) - mass downloading entire website of artwork

Hey peeps, howdy, hope you're all keeping safe :)

I'm in the process of setup a 4K smart mirror with raspberry pi in the house, and would like to have it display a bunch of photos as a slideshow. I have been on various sites and doing the acquisition process manually and it's a pain in the ass. Is there an easy way I can strip an entire site (deviantart/behanced/artstation/ello/etc) of a particular category of artwork with requirement of 1) subject of photography/cgi only; and 2) 4K or nearest to 4K only, no duplicates, so I don't have 4/5/6/8K of the same piece.

Whilst I can RSS randos off a site but then if something cool comes up I'd like to keep a copy and that's additional work. Ideally I'd like to get an one off of 1,000,000-1,500,000 images, then spend 2 weeks removing ones I don't like instead of acquiring them one at a time.

Storage/nbn bandwidth not an issue, and personal consumption not commercial in case it's a topic of contention.

Many thanks in advance!

Comments

  • I've used Webzip years ago, but you may find security adn coding blocks it these days

    search for offline browsers

  • I assume you've googled for: deviantart image downloader

    I didn't check all the results, but I'm sure this question has been asked a few times already.

  • +2

    I wrote a script to download the top images from r/earthporn. It runs everyday and the collection goes up by 30-40 a day. The subreddit pages all provide rss feeds.

    The other option is unsplash which allows filtering by attributes.

    Last option, torrent a good collection. Plenty on TPB.

Login or Join to leave a comment