I always prefer to self-host my dependencies, but as a developer who prefer to avoid an npm-based webpack/whatever build pipeline it's often WAY harder to do that than I'd like.
If you are the developer of an open source JavaScript library, please take the time to offer a downloadable version of it that works without needing to run an "npm install" and then fish the right pieces out of the node_modules folder.
jQuery still offer a single minified file that I can download and use. I wish other interesting libraries would do the same!
(I actually want to use ES Modules these days which makes things harder due to the way they load dependencies. I'm still trying to figure out the best way to use import maps to solve this.)
The assumption of many npm packages is that you have a bundler and I think rightly so because that leaves all options open regarding polyfilling, minification and actual bundling.
I would agree with you if minification delivered marginal gains, but it will generally roughly halve the size of a large bundle or major JS library (compared to just gzip'ing it alone), and this is leaving aside further benefits you can get from advanced minification with dead code removal and tree-shaking. That means less network transfer time and less parse time. At least for my use-cases, this will always justify the extra build step.
I really miss the days of minimal/no use of JS in websites (not that I want java-applets and Flash LOL). Kind of depressing that so much of the current webdesign is walled behind javascript.
Cool, I can download 20 MB of JavaScript instead of 40. Everyone uses minification, and "web apps" still spin up my laptop fans. Maybe we've lost the plot.
There might be a negative incentive in play: you may be compressing packages, but having your dependencies available at the tip of *pm install bloats overall size and complexity beyond what lack of bundling would give you.
The assumption shouldn't be that you have a bundler, but that your tools and runtimes support standard semantics, so you can bundle if you want to, or not bundle if you don't want to.
Ime this has always been standard practice for production code at all the companies I've worked at and with as a SWE or PM - store dependencies within your own internal Artifactory, have it checked by a vuln scanner, and then called and deployed.
That said, I came out of the Enterprise SaaS and Infra space so maybe workflows are different in B2C, but I didn't a difference in the customer calls I've been on.
I guess my question is why your employer or any other org would not follow the model above?
> I guess my question is why your employer or any other org would not follow the model above?
Frankly, it's because many real-world products are pieced together by some ragtag group of bright people who have been made responsible for things they don't really know all that much about.
The same thing that makes software engineering inviting to autodidacts and outsiders (no guild or license, pragmatic 'can you deliver' hiring) means that quite a lot of it isn't "engineered" at all. There are embarrassing gaps in practice everywhere you might look.
Yep. The philosophy most software seems to be written with is “poke it until it works locally, then ship it!”. Bugs are things you react to when your users complain. Not things you engineer out of your software, or proactively solve.
This works surprisingly well. It certainly makes it easier to get started in software. Well, so long as you don’t mind that most modern software performs terribly compared to what the computer is capable of. And suffers from reliability and security issues.
Counterpoint: It's not not about being an autodidact or an outsider.
I was unlikely to meet any bad coders at work, due to how likely it is they were filtered by the hiring process, and thus I never met anyone writing truly cringe-worthy code in a professional setting.
That was until I decided to go to university for a bit[1]. This is where, for the first time, I met people writing bad code professionally: professors[2]. "Bad" as in best-practices, the code usually worked. I've also seen research projects that managed to turn less than 1k LOC of python into a barely-maintainable mess[3].
I'll put my faith in an autodidact who had to prove themselves with skills and accomplishments alone over someone who got through the door with a university degree.
An autodidact who doesn't care about their craft is not going to make the cut, or shouldn't. If your hiring process doesn't filter those people, why are you wasting your time at a company that probably doesn't know your value?
[1] Free in my country, so not a big deal to attend some lectures besides work. Well, actually I'm paying for it with my taxes, so I might as well use it.
[2] To be fair, the professors teaching in actual CS subjects were alright. Most fields include a few lectures on basic coding though, which were usually beyond disappointing. The non-CS subject that had the most competent coders was mathematics. Worst was economics. Yes, I meandered through a few subjects.
[3] If you do well on some test you'd usually get job offers from professors, asking you to join their research projects. I showed up to interviews out of interest in the subject matter and professors are usually happy to tell you all about it, but wages for students are fixed at the legal minimum wage, so it couldn't ever be a serious consideration for someone already working on the free market.
I was hoping ongoing coverage would answer that; it sounds like a perfect example. I heard that the tampered code redirects traffic to a sports betting site.
There are cheaper or free alternatives to Artifactory. Yes they may not have all of the features but we are talking about a company that is fine with using a random CDN instead.
Or, in the case of javascript, you could just vendor your dependencies or do a nice "git add node_modules".
I just gave Artifactory as an example. What about GHE, self-hosted GitLab, or your own in-house Git?
Edit: was thinking - would be a pain in the butt to manage. That tracks, but every org ik has some corporate versioning system that also has an upsell for source scanning.
I've been a part of a team which had to manage a set of geodistributed Artifactory clusters and it was a pain in the butt to manage, too - but these were self-hosted. At a certain scale you have to pick the least worst solution though, Artifactory seems to be that.
This is kinda sad. For introducing new dependencies, a vuln scanner makes sense (don't download viruses just because they came from a source checkout!), but we could have kept CDNs if we'd used signatures.
EDIT: Never mind, been out of the game for a bit! I see there is SRI now...
This supply chain attack had nothing to do with npm afaict.
The dependency in question seems to be (or claim to be) a lazy loader that determines browser support for various capabilities and selectively pulls in just the necessary polyfills; in theory this should make the frontend assets leaner.
But the CDN used for the polyfills was injecting malicious code.
I would expect latency (network round trip time) to make this entire exercise worthless. Most polyfills are 1kb or less. Splitting polyfill code amongst a bunch of small subresources that are loaded from a 3rd party domain sounds like it would be a net loss to performance. Especially since your page won’t be interactive until those resources have downloaded.
Your page will almost certainly load faster if you just put those polyfills in your main js bundle. It’ll be simpler and more reliable too.
In practice when this wasn't a Chinese adware service, it proved to be faster to use the CDN.
You are not loading a "bunch" of polyfill script files, you selected what you needed in the URL via a query parameter, and the service took that plus user agent of the request to determine which were needed and returned a minified file of just the necessary polyfills.
As this request was to a separate domain it did not run into the head of line / max connections per domain issue of Http 1.1 which was still the more common protocol at the time this service came out.
Js dependencies should be pretty small compared to images or other resources. Http pipelining should make it fast to load them from your server with the rest
The only advantage to using one of those cdn-hosted versions is that it might help with browser caching
> Http pipelining should make it fast to load them from your server with the rest
That's true, but it should be emphasized that it's only fast if you bundle your dependencies, too.
Browsers and web developers haven't been able to find a way to eliminate a ~1ms/request penalty for each JS file, even if the files are coming out of the local cache.
If you're making five requests, that's fine, but if you're making even 100 requests for 10 dependencies and their dependencies, there's a 100ms incentive to do at least a bundle that concatenates your JS.
And once you've added a bundle step, you're a few minutes away from adding a bundler that minifies, which often saves 30% or more, which is usually way more than you probably saved from just concatenating.
> The only advantage to using one of those cdn-hosted versions is that it might help with browser caching
And that is not true. Browsers have separate caches for separate sites for privacy reasons. (Before that, sites could track you from site to site by seeing how long it took to load certain files from your cache, even if you'd disabled cookies and other tracking.)
There is still a caching effect of the CDN for your servers, even if there isn't for the end user: if the CDN serves the file then your server does not have to.
Large CDNs with endpoints in multiple locations internationally also give the advantage of reducing latency: if your static content comes from the PoP closest to me (likely London, <20ms away where I'm currently sat, ~13 on FTTC at home⁰, ~10 at work) that could be quite a saving if your server is otherwise hundreds of ms away (~300ms for Tokyo, 150 for LA, 80 for New York). Unless you have caching set to be very aggressive dynamic content still needs to come from your server, but even then a high-tech CDN can² reduce the latency of the TCP connection handshake and¹ TLS handshake by reusing an already open connection between the CDN and the backing server(s) to pipeline new requests.
This may not be at all important for many well-designed sites, or sites where latency otherwise matters little enough that a few hundred ms a couple of times here or there isn't really going to particularly bother the user, but could be a significant benefit to many bad setups and even a few well-designed ones.
--------
[0] York. The real one. The best one. The one with history and culture. None of that “New” York rebranded New Amsterdam nonsense!
[1] if using HTTPS and you trust the CDN to re-encrypt, or HTTP and have the CDN add HTTPS, neither of which I wouldn't recommend as it is exactly an MitM situation, but both are often done
[2] assuming the CDN also manages your DNS for the whole site, or just a subdomain for the static resources, so the end user sees the benefit of the CDNs anycast DNS arrangement.
I don't want a build pipeline. I want to write some HTML with a script type=module tag in it with some JavaScript, and I want that JavaScript to load the ES modules it depends on using import statements (or dynamic import function calls for lazy loading).
Do you not use CSS preprocessors or remote map files or anything like that... or do you just deal with all of that stuff manually instead of automating it?
I suspect this is more relevant for people who aren't normally JavaScript developers. (Let's say you use Go or Python normally.) It's a way of getting the benefits of multi-language development while still being mostly in your favorite language's ecosystem.
On the Node.js side, it's not uncommon to have npm modules that are really written in another language. For example, the esbuild npm downloads executables written in Go. (And then there's WebAssembly.)
In this way, popular single-language ecosystems evolve towards becoming more like multi-language ecosystems. Another example was Python getting 'wheels' straightened out.
So the equivalent for bringing JavaScript into the Python ecosystem might be having Python modules that adapt particular npm packages. Such a module would automatically generate JavaScript based on a particular npm, handling the toolchain issue for you.
A place to start might be a Python API for the npm command itself, which takes care of downloading the appropriate executable and running it. (Or maybe the equivalent for Bun or Deno?)
This is adding still more dependencies to your supply chain, although unlike a CDN, at least it's not a live dependency.
Sooner or later, we'll all depend on left-pad. :-)
This is why I don't think it's very workable to avoid npm. It's the package manager of the ecosystem, and performs the job of downloading dependencies well.
I personally never want to go back to the pre-package-manager days for any language.
One argument is that Javascript-in-the-browser has advanced a lot and there's less need for a build system. (ex. ESM module in the browser)
I have some side projects that are mainly HTMX-based with some usage of libraries like D3.js and a small amount of hand-written Javascript. I don't feel that bad about using unpkg because I include signatures for my dependencies.
Before ESM I wasn't nearly as sold on skipping the build step, but now it feels like there's a much nicer browser native way of handling dependencies, if only I can get the files in the right shape!
npm is a package manager though, not a build system. If you use a library that has a dependency on another library, npm downloads the right version for you.
Yep. And so does unpkg. If you’re using JavaScript code through unpkg, you’re still using npm and your code is still bundled. You’re just getting someone else to do it, at a cost of introducing a 3rd party dependency.
I guess if your problem with npm and bundlers is you don’t want to run those programs, fine? I just don’t really understand what you gain from avoiding running bundlers on your local computer.
Oh lol yeah, I recently gave up and just made npm build part of my build for a hobby project I was really trying to keep super simple, because of this. It was too much of a hassle to link in stuff otherwise, even very minor small things
You shouldn't need to fish stuff out of node_moduoes though, just actually get it linked and bundled into one is so that it automatically grabs exactly what you need and it's deps.
If this process sketches you out as it does me, one way to address that, as I do, is have the bundle emitted with minification disabled so its easy to review
If you are the developer of an open source JavaScript library, please take the time to offer a downloadable version of it that works without needing to run an "npm install" and then fish the right pieces out of the node_modules folder.
jQuery still offer a single minified file that I can download and use. I wish other interesting libraries would do the same!
(I actually want to use ES Modules these days which makes things harder due to the way they load dependencies. I'm still trying to figure out the best way to use import maps to solve this.)