How to setup private npm registry
Why we might need one?
There can be scenarios where we have some code which is used by many micro-services. For example, we can have a code on how to make interservice calls with the circuit breaker, logging, timeouts and much more. We want every service in our infrastructure to adhere to same guidelines. It would be great if we could install these as npm packages and version manage the changes. This will give more confidence in rolling out new and breaking changes thus enabling higher velocity in the product development.
We will be using Verdaccio as a npm registry. From its
Verdaccio is a simple, zero-config-required local private npm registry. No need for an entire database just to get started! Verdaccio comes out of the box with its own tiny database, and the ability to proxy other registries (eg. npmjs.org), caching the downloaded modules along the way.
How to setup Verdaccio
We would be setting this up on AWS Ubuntu EC2 instance. We will be using docker version for ease of installation. Install
docker on the instance.
We would be starting our verdaccio docker image via docker-compose. Install docker-compose
In case vim is not installed, install vim
Now create a folder where we would write our docker-compose file. Let’s call is
myVerdaccio and inside this folder create a
Two things to note here. First in
line 14 where we are mapping our local folder to docker folder. We want our storage to persist even if we remove the docker image. Second we are mapping verdaccio port to port 80.
Now the only thing we need is that our docker images should be able to write to the
storage folder. Create a folder
myVerdaccio folder, and set the right access to this folder. For the demo purpose, we can just do
chmod -R 777 storage .
docker-compose up and we have Verdaccio up and running!! Now if we go to the ec2 url, we would be able to see the verdaccio UI something like below
How to use private npm registry
Now we can go to any of our code repository, create a new file at the root of the folder called
.npmrc and have its content as follows
npm install and all packages will be installed.
How do I know if my private registry is used and not a public one
Good point! There are many ways:
npm config get registrywill give the url of the registry it is going to use
- We can look at the docker logs to verify we are getting request when we do npm install
- We can look at
myVerdaccio/storage/datain the ec2 instance and we will find packages that we have just installed.
There is nothing private about it!
Some of the astute reader would have already spotted that. There is no authentication. Verdaccio has documentation on setting up authentication and a bunch of plugins as well. We will use htpasswd which is built into Verdaccio. Here are the requirements for which we are building
- Only authorised users should be able to read and write the packages.
- User creation can only be done by admin.
Create a folder
myVerdaccio/conf and write a config file. We will use the default config verdaccio file as a base and make changes
Few things to note in this file:
max_users: -1which means users cannot be created
Line 45 and 49: mentions that only authenticated users can read and write to this registry
htpasswd file where username and password are stored (in this example, both are
We can create a different file by going to htaccesstools and generate a new username/password combination.
Next step is to tell the docker to use this new conf file. Update the
docker-compose.yml file to reflect the same
Let’s put this to test.
- restart docker
rm -rf node modulesand
rm package-lock.jsonin the repo source repo
If things go right, we would see the error like above.
npm login --registry http://ec2-ip.ap-south-1.compute.amazonaws.com
Email: (this IS public) firstname.lastname@example.org
Logged in as verdaccio on http://ec2-ip.ap-south-1.compute.amazonaws.com/.
This creates an access token. Token is created at
~/.npmrc which would look something like this
npm install will work.
How to publish?
It’s actually pretty simple. Just install np. It does all the magic. As long as we are logged in, there is nothing we have to do. Heck, it will push the code for us in the source repo as well.
How to handle high availability and increased workloads?
Now what happens if this docker goes down? What if docker image is not able to handle the load? Let’s horizontally scale our system. However there are few challenges:
- Each docker images writes to its local mounted space. We want to share the storage across ec2 machines.
- How to make sure when login happens on one machine, it persists for all the systems behind the load balancer.
To solve the first problem, we will use AWS Elastic File System for sharing storage across EC2 machines. I do not have the right answer to the second problem. It looks like this is across various plugins. As an experiment, I was trying to use bitbucket auth plugin but still has the same issue. I have raised an issue for the same. As a work around, we just login once to the box manually. This is how our design looks like
Let’s look at some of the code changes we will have to make.
First, let’s create a new folder, called
efs-storage . Once we have created efs in AWS, we can mount with the below command
mount -t nfs4 -o nfsvers=4.1,rsize=1048576,wsize=1048576,hard,timeo=600,retrans=2,noresvport fs-<number>.efs.ap-south-1.amazonaws.com:/ /home/ubuntu/myVerdaccio/efs-storage
However if we want this mount to persist over system restart, we need to add it to
fs-<number>.efs.ap-south-1.amazonaws.com:/ /home/ubuntu/myVerdaccio/efs-storage nfs4 nfsvers=4.1,rsize=1048576,wsize=1048576,hard,timeo=600,retrans=2,noresvport 0 0
We can try if this is working by running
mount -a and then doing
df -h , we would see an entry for the mount point pointing to efs.
So, now we have our very own production ready npm registry.
If you found this story interesting or useful, please support it by clapping it👏 .