![image.png](https://ipfs.busy.org/ipfs/QmWZEyU85cKJKegJshCGqLdVrArpWBgtXwPmXoyyuxGnzr)


![image.png](https://ipfs.busy.org/ipfs/QmSYwbLBSwpbfVAj4BFg3ykUDmXg2nUko8HwhbRLdjYg3S)

I spent a couple hours developing a bit of code because I wanted to download all of the responsive CSS templates on free-css.com. It was a very pleasant experience. Coding, scraping, so much love. ♥♥

I saved my code in a Gist in case you want to  see it (or use it, or modify it):

https://gist.github.com/CryptoSharon/f0face4028b6e3cd1a32e519a0ea68ac

Nananananana, still scraping.

![image.png](https://ipfs.busy.org/ipfs/QmQcMn47hocgeS4V4vRD7Btpjx9enDYvVU6YFJELotMB3S)

I would have run it on my server to share the big file on here, but I wasn't thinking ahead or planning to share it. Hmmm. I guess if you want it, you'll have to scrape it too? That would be mean to free-css, so much good work and we're spamming their website with scrapers.

<h2>About the code</h2>

I'm using a few cool libraries that I love, i.e.: cheerio, request-promise, bluebird and fs-extra. The traditional way would have been using just cheerio, request and fs, but I love promisifying everything. I also like the extra functionalities that fs.extra provides! The `fs.pathExists(path)` is wonderful. The traditional `fs` library wanted me to do some weird `fs.access(path)` thingy and then use a property of one of the various results given? Ugly solution.

I also learned that Cheerio apparently doesn't have the same `.attr('href')` as jQuery. Instead, I had to run `$('a').attribs.href`. After about half an hour of trying out things in jQuery, then trying them out on Cheerio and comparing the errors and logging the results, I managed to properly grab each URL link (12 per page) to add it to the array that would be downloaded in the end.

I also discovered that I can make a delayed loop with this wonderful, wonderful function!

```js
const awaitForEach = async (fn, arr) => {
	for (const el of arr) {
		await fn(el)
	}
}
```

Props to [this blogger](https://blog.lavrton.com/javascript-loops-how-to-handle-async-await-6252dd3c795) for showing me how to do that.

I was doing everything functionally, but I got very sleepy, bored and tired when I got to some unexpected errors and the `getZip()` function ended up imperative. Sad ¯\\\_(ツ)_/¯, but I was getting impatient after 2 hours of coding pretty things.

Another thing to note is that not all of the files are straightforwardly downloaded. Some files, when you request them, tell you that they're not available, but that there are similar files that are available. It gives you options. I had to create this tiny piece of code to handle this Error Status 300, download files and continue with the rest of the stack.

```js
.catch(e => {
	if (e.statusCode === 300) {
		const $ = cheerio.load(e.error)
		const links = $('li a').map((i, el) =>
			({ zip_link: baseUrl + el.attribs.href }))
		downloadAllLinks(Array.from(links))
	} else {
		console.error(e)
	}
})
```

I also discovered that `$().map` doesn't return an iterable array! It's like `{0: value1, 1: value2, ...randomObject}`. It has 0, 1, 2, etc., so I could basically do `Array.from(cheerioMappedArray)` and get a valid array without object thingies so that it could be iterated by my `awaitForEach()`. I explored a few other options but the normal `.map` and ramda's `map` also attach the strange object to the array. Only `Array.from` could create a shallow clone taking only the iterable properties.

<h2>File size average</h2>

Okay! The files are done downloading. 658 files, 1.72 GB.
1.72*1024 = 1761.28
1761.28/658 = 2.676

The average size of the zipped templates is 2.6 megabytes. Not big, not small. I hope to find a use for all of this. I would be a good [DataHoarder](https://www.reddit.com/r/DataHoarder/) if I had any money to buy HDD's and SSD's. Instead, I'll probably have to delete it in a few months to get space for other scrapes and downloads.

Alrighty then, this was fun.