source

How to setup private npm registry

Why we might need one?

There can be scenarios where we have some code which is used by many micro-services. For example, we can have a code on how to make interservice calls with the circuit breaker, logging, timeouts and much more. We want every service in our infrastructure to adhere to same guidelines. It would be great if we could install these as npm packages and version manage the changes. This will give more confidence in rolling out new and breaking changes thus enabling higher velocity in the product development.

We will be using Verdaccio as a npm registry. From its readme.md

Verdaccio is a simple, zero-config-required local private npm registry. No need for an entire database just to get started! Verdaccio comes out of the box with its own tiny database, and the ability to proxy other registries (eg. npmjs.org), caching the downloaded modules along the way.

How to setup Verdaccio

We would be setting this up on AWS Ubuntu EC2 instance. We will be using docker version for ease of installation. Install git, docker on the instance.

We would be starting our verdaccio docker image via docker-compose. Install docker-compose

Install docker compose

In case vim is not installed, install vim

install vim on ubuntu

Now create a folder where we would write our docker-compose file. Let’s call is myVerdaccio and inside this folder create a docker-compose.yml file

docker-compose file for Verdaccio

Two things to note here. First inline 14 where we are mapping our local folder to docker folder. We want our storage to persist even if we remove the docker image. Second we are mapping verdaccio port to port 80.

Now the only thing we need is that our docker images should be able to write to the storage folder. Create a folder storage inside myVerdaccio folder, and set the right access to this folder. For the demo purpose, we can just do chmod -R 777 storage .

Run docker-compose up and we have Verdaccio up and running!! Now if we go to the ec2 url, we would be able to see the verdaccio UI something like below

Verdaccio home screen

How to use private npm registry

Now we can go to any of our code repository, create a new file at the root of the folder called .npmrc and have its content as follows

Run npm install and all packages will be installed.

How do I know if my private registry is used and not a public one

Good point! There are many ways:

1. npm config get registry will give the url of the registry it is going to use
2. We can look at the docker logs to verify we are getting request when we do npm install
3. We can look at myVerdaccio/storage/data in the ec2 instance and we will find packages that we have just installed.

There is nothing private about it!

Some of the astute reader would have already spotted that. There is no authentication. Verdaccio has documentation on setting up authentication and a bunch of plugins as well. We will use htpasswd which is built into Verdaccio. Here are the requirements for which we are building

• Only authorised users should be able to read and write the packages.
• User creation can only be done by admin.

Create a folder myVerdaccio/conf and write a config file. We will use the default config verdaccio file as a base and make changes

Few things to note in this file:

• Line 22 has max_users: -1 which means users cannot be created
• Line 45 and 49 : mentions that only authenticated users can read and write to this registry

Create our htpasswd file where username and password are stored (in this example, both are verdaccio )

We can create a different file by going to htaccesstools and generate a new username/password combination.

Next step is to tell the docker to use this new conf file. Update the docker-compose.yml file to reflect the same

Let’s put this to test.

• restart docker
• rm -rf node modules and rm package-lock.json in the repo source repo
• npm install

Verdaccio authentication failed error

If things go right, we would see the error like above.

Let’s login

npm login --registry http://ec2-ip.ap-south-1.compute.amazonaws.com
Username: verdaccio
Password:
Email: (this IS public) test@test.com
Logged in as verdaccio on http://ec2-ip.ap-south-1.compute.amazonaws.com/.

This creates an access token. Token is created at ~/.npmrc which would look something like this

//ec2-ip.ap-south-1.compute.amazonaws.com/:_authToken=<authtoken>

Now npm install will work.

How to publish?

It’s actually pretty simple. Just install np. It does all the magic. As long as we are logged in, there is nothing we have to do. Heck, it will push the code for us in the source repo as well.

How to handle high availability and increased workloads?

Now what happens if this docker goes down? What if docker image is not able to handle the load? Let’s horizontally scale our system. However there are few challenges:

• Each docker images writes to its local mounted space. We want to share the storage across ec2 machines.
• How to make sure when login happens on one machine, it persists for all the systems behind the load balancer.

To solve the first problem, we will use AWS Elastic File System for sharing storage across EC2 machines. I do not have the right answer to the second problem. It looks like this is across various plugins. As an experiment, I was trying to use bitbucket auth plugin but still has the same issue. I have raised an issue for the same. As a work around, we just login once to the box manually. This is how our design looks like

High availability for npm registry

Let’s look at some of the code changes we will have to make.

First, let’s create a new folder, called efs-storage . Once we have created efs in AWS, we can mount with the below command

mount -t nfs4 -o nfsvers=4.1,rsize=1048576,wsize=1048576,hard,timeo=600,retrans=2,noresvport fs-<number>.efs.ap-south-1.amazonaws.com:/ /home/ubuntu/myVerdaccio/efs-storage

However if we want this mount to persist over system restart, we need to add it to /etc/fstab

fs-<number>.efs.ap-south-1.amazonaws.com:/ /home/ubuntu/myVerdaccio/efs-storage nfs4 nfsvers=4.1,rsize=1048576,wsize=1048576,hard,timeo=600,retrans=2,noresvport 0 0

We can try if this is working by running mount -a and then doing df -h , we would see an entry for the mount point pointing to efs.

So, now we have our very own production ready npm registry.

If you found this story interesting or useful, please support it by clapping it👏 .