How to setup private npm registry
Why we might need one?
There can be scenarios where we have some code which is used by many micro-services. For example, we can have a code on how to make interservice calls with the circuit breaker, logging, timeouts and much more. We want every service in our infrastructure to adhere to same guidelines. It would be great if we could install these as npm packages and version manage the changes. This will give more confidence in rolling out new and breaking changes thus enabling higher velocity in the product development.
We will be using Verdaccio as a npm registry. From its readme.md
Verdaccio is a simple, zero-config-required local private npm registry. No need for an entire database just to get started! Verdaccio comes out of the box with its own tiny database, and the ability to proxy other registries (eg. npmjs.org), caching the downloaded modules along the way.
How to setup Verdaccio
We would be setting this up on AWS Ubuntu EC2 instance. We will be using docker version for ease of installation. Install git
, docker
on the instance.
We would be starting our verdaccio docker image via docker-compose. Install docker-compose
In case vim is not installed, install vim
Now create a folder where we would write our docker-compose file. Let’s call is myVerdaccio
and inside this folder create a docker-compose.yml
file
Two things to note here. First inline 14
where we are mapping our local folder to docker folder. We want our storage to persist even if we remove the docker image. Second we are mapping verdaccio port to port 80.
Now the only thing we need is that our docker images should be able to write to the storage
folder. Create a folder storage
inside myVerdaccio
folder, and set the right access to this folder. For the demo purpose, we can just do chmod -R 777 storage
.
Run docker-compose up
and we have Verdaccio up and running!! Now if we go to the ec2 url, we would be able to see the verdaccio UI something like below
How to use private npm registry
Now we can go to any of our code repository, create a new file at the root of the folder called .npmrc
and have its content as follows
Run npm install
and all packages will be installed.
How do I know if my private registry is used and not a public one
Good point! There are many ways:
npm config get registry
will give the url of the registry it is going to use- We can look at the docker logs to verify we are getting request when we do npm install
- We can look at
myVerdaccio/storage/data
in the ec2 instance and we will find packages that we have just installed.
There is nothing private about it!
Some of the astute reader would have already spotted that. There is no authentication. Verdaccio has documentation on setting up authentication and a bunch of plugins as well. We will use htpasswd which is built into Verdaccio. Here are the requirements for which we are building
- Only authorised users should be able to read and write the packages.
- User creation can only be done by admin.
Create a folder myVerdaccio/conf
and write a config file. We will use the default config verdaccio file as a base and make changes
Few things to note in this file:
Line 22
hasmax_users: -1
which means users cannot be createdLine 45 and 49
: mentions that only authenticated users can read and write to this registry
Create our htpasswd
file where username and password are stored (in this example, both are verdaccio
)
We can create a different file by going to htaccesstools and generate a new username/password combination.
Next step is to tell the docker to use this new conf file. Update the docker-compose.yml
file to reflect the same
Let’s put this to test.
- restart docker
rm -rf node modules
andrm package-lock.json
in the repo source reponpm install
If things go right, we would see the error like above.
Let’s login
npm login --registry http://ec2-ip.ap-south-1.compute.amazonaws.com
Username: verdaccio
Password:
Email: (this IS public) test@test.com
Logged in as verdaccio on http://ec2-ip.ap-south-1.compute.amazonaws.com/.
This creates an access token. Token is created at ~/.npmrc
which would look something like this
//ec2-ip.ap-south-1.compute.amazonaws.com/:_authToken=<authtoken>
Now npm install
will work.
How to publish?
It’s actually pretty simple. Just install np. It does all the magic. As long as we are logged in, there is nothing we have to do. Heck, it will push the code for us in the source repo as well.
How to handle high availability and increased workloads?
Now what happens if this docker goes down? What if docker image is not able to handle the load? Let’s horizontally scale our system. However there are few challenges:
- Each docker images writes to its local mounted space. We want to share the storage across ec2 machines.
- How to make sure when login happens on one machine, it persists for all the systems behind the load balancer.
To solve the first problem, we will use AWS Elastic File System for sharing storage across EC2 machines. I do not have the right answer to the second problem. It looks like this is across various plugins. As an experiment, I was trying to use bitbucket auth plugin but still has the same issue. I have raised an issue for the same. As a work around, we just login once to the box manually. This is how our design looks like
Let’s look at some of the code changes we will have to make.
First, let’s create a new folder, called efs-storage
. Once we have created efs in AWS, we can mount with the below command
mount -t nfs4 -o nfsvers=4.1,rsize=1048576,wsize=1048576,hard,timeo=600,retrans=2,noresvport fs-<number>.efs.ap-south-1.amazonaws.com:/ /home/ubuntu/myVerdaccio/efs-storage
However if we want this mount to persist over system restart, we need to add it to /etc/fstab
fs-<number>.efs.ap-south-1.amazonaws.com:/ /home/ubuntu/myVerdaccio/efs-storage nfs4 nfsvers=4.1,rsize=1048576,wsize=1048576,hard,timeo=600,retrans=2,noresvport 0 0
We can try if this is working by running mount -a
and then doing df -h
, we would see an entry for the mount point pointing to efs.
So, now we have our very own production ready npm registry.
If you found this story interesting or useful, please support it by clapping it👏 .