Use NestJS, MongoDB and Docker to Create an URL Shortener

This tutorial will cover all work to build a simple URL shortener API with NestJS and MongoDB. Additionally, I will show how to deploy it to a (Docker and Docker Swarm) production environment using Docker.

The source code is published on GitHub and can be used freely:

GitHub - paulknulst/paulsshortener

Contribute to paulknulst/paulsshortener development by creating an account on GitHub.

GitHubpaulknulst

Create the NestJS Project

First, create a NestJS project which will work as a baseline for the URL shortener. To do this, you need to install the Nest-CLI on your system which can be done by:

npm install -g @nestjs/cli

Then switch to your projects folder and use the Nest-CLI to create a new NestJS project by executing:

nest new paulsshortener

During this process, you will be asked which package manager you want to use. Within this tutorial npm will be used.

Now, you can start the NestJS project in "watch-mode" to instantly see all changes you will make during this tutorial by executing:

npm run start:dev

To test if everything is started correctly hit http://localhost:3000 which will just show "Hello World" within the browser.

Within the project open the AppService (/src/app.service.ts) and change return 'Hello World!'; to return 'This will be your URL shortener';. After reloading http://localhost:3000 the updated response will be seen.

Set Up MongoDB database

For simplicity, MongoDB will be used during this tutorial. To avoid trouble setting up the correct version, replacing an old installation, and so on you should use Docker to deploy it. Save this minimal Compose file into your project root and name it docker-compose.local.yml:

version: '3.6'

services:
  mongo:
    image: mongo
    restart: always
    environment:
      MONGO_INITDB_ROOT_USERNAME: root
      MONGO_INITDB_ROOT_PASSWORD: supersafe

If you are not familiar with Docker Compose and do not want to use it to set up a MongoDB you can have a look at the official MongoDB documentation to learn how you can install it on your machine. Keep in mind that Docker is also used to deploy this URL shortener later within this tutorial!

Now, to deploy a MongoDB, switch to your project root within a terminal and execute:

docker-compose -f docker-compose.local.yml up -d

Afterward, you have successfully set up MongoDB and your machine and can start using it.

Connect Your NestJS project to MongoDB

To connect the project to MongoDB you should use Mongoose which is the most popular MongoDB object modeling tool. Start by installing the required dependencies into your project:

npm i @nestjs/mongoose mongoose

Once you have installed the dependency you can import the Mongoose module into the project by editing the AppModule (/src/app.module.ts) that it looks like this:

import { Module } from '@nestjs/common';
import { AppController } from './app.controller';
import { AppService } from './app.service';
import { MongooseModule } from '@nestjs/mongoose';

@Module({
  imports: [MongooseModule.forRoot('mongodb://localhost:12345/paulsshortener')],
  controllers: [AppController],
  providers: [AppService],
})

export class AppModule {}

The connection will now be automatically established and you can create the schema that will be used to work with the database.

Now use the Nest-CLI to create a new module within your NestJS project which will handle everything related to URLs:

nest g res url

The routine will ask two questions that you should answer.

What transport layer do you use? -> REST API
Would you like to generate CRUD entry points? -> No

Switch to the newly generated url folder, create a new folder schema, and add a file url.schema.ts. Then create a class (Url) add two properties (url, shortenedUrl) to the file. Also, add exports for Document and Schema. Your file should look like this:

import { Prop, Schema, SchemaFactory } from '@nestjs/mongoose';
import { Document } from 'mongoose';

export type UrlDocument = Url & Document;

@Schema()
export class Url {
  @Prop()
  url: string;

  @Prop()
  shortenedUrl: string;
}

export const UrlSchema = SchemaFactory.createForClass(Url);

Open up url.module.ts, add two new imports, and modify the @Module annotation:

import {Module} from '@nestjs/common';
import {UrlService} from './url.service';
import {UrlController} from './url.controller';
import {Url, UrlSchema} from "./schemas/url.schema";
import {MongooseModule} from "@nestjs/mongoose";

@Module({
    imports: [MongooseModule.forFeature([{name: Url.name, schema: UrlSchema}])],
    controllers: [UrlController],
    providers: [UrlService]
})
export class UrlModule {
}

Implement the URL Link Shortner Function

To shrink an URL you will be going to use the CRC32 hash algorithm which has to be added to the project to use it:

npm install crc-32 --save

Open the UrlService (/src/url/url.service.ts) and replace the content of the file with the following snippet:

import {Injectable} from '@nestjs/common';
import {InjectModel} from "@nestjs/mongoose";
import {Url, UrlDocument} from "./schemas/url.schema";
import {Model} from "mongoose";
import * as CRC from "crc-32";

@Injectable()
export class UrlService {

    constructor(@InjectModel(Url.name) private urlModel: Model<UrlDocument>) {
    }

    private shrink(url: string) {
        return CRC.str(url).toString(16)
    }

    async create(url: string) {
        const createdUrl = new this.urlModel({url: url, shortenedUrl: this.shrink(url)});
        await createdUrl.save();
        return createdUrl.shortenedUrl;
    }

    async find(shortenedUrl: string) { //-3c666ac
        const url = await this.urlModel.findOne({shortenedUrl: shortenedUrl}).exec();
        return url.url;
    }
}

This snippet contains three important functions:

shrink: Converts the given URL into an 8-character string by using the CRC32 algorithm. Then returns only the 8-character string.
create: Use the shrink function to create an 8-character string and saves a new UrlSchema document into the MongoDB
find: Retrieves the saved URL from the MongoDB.

Add Routes to NestJS API

To use the URL shortener in a Client we need to create three REST endpoints/functions.

GET /shrink: Create a new shortened URL using the HTTP Get method. The path parameter contains the unshortened URL. This can be done within any browser.
POST /shrink: Create a new shortened URL using the HTTP Post method. The body contains the unshortened URL. You need an API or Postman to use this.
GET /s: Takes the 8-character string as a path parameter and returns the unshortened URL. In the end, it will automatically forward to the unshortened URL.

As you are working with NestJS you can easily implement these endpoints by adding three new functions to the UrlController (url.controller.ts) within the URL module and annotate it with @Get and @Post. Open the UrlController and replace the content with:

import {Body, Controller, Get, Param, Post} from '@nestjs/common';
import {UrlService} from './url.service';

@Controller('')
export class UrlController {
    constructor(private readonly urlService: UrlService) {
    }

    @Get('/shrink/:url')
    getShrink(@Param('url') url: string) {
        return this.urlService.create(url)
    }

    @Post('/shrink')
    postShrink(@Body() body: { url: string }) {
        return this.urlService.create(body.url)
    }

    @Get('/s/:shortenedUrl')
    unshrink(@Param('shortenedUrl') shortenedUrl: string) {
        return this.urlService.find(shortenedUrl)
    }
}

Within this snippet, two new GET resources (/shrink/ and /s/) and one POST resource that calls the previously created functions from the UrlService are created. Also, the annotation above the class is changed from @Controller('url') to @Controller('') to further shrink the resulting URL.

Testing the URL shortener

To test the functionality you can simply use your browser because we have implemented both resources as GET calls.

Keep in mind that as you use GET for providing an URL as a path parameter you have to encode the URL! This means that an URL like this:

https://www.knulst.de/manage-time-more-efficiently-with-the-pomodoro-technique/

will become this:

https%3A%2F%2Fwww.knulst.de%2Fmanage-time-more-efficiently-with-the-pomodoro-technique%2F

With this information, you can create a shortened URL by opening the following URL in our browser:

http://localhost:3000/shrink/https%3A%2F%2Fwww.knulst.de%2Fmanage-time-more-efficiently-with-the-pomodoro-technique%2F

With this GET call your API will return the 8-character string: -5f1a8349 (It should be the same within your project)

Append this string to your GET /s/ call to receive the unshortened version of the URL by opening: http://localhost:3000/s/-5f1a8349

The result in the browser will be the unshortened version of the previously provided URL.

Implement Forwarding to Unshortened URL

Now, that you have developed an API that can shorten URLs with help of the CRC-32 algorithm you should enable the functionality to forward the request and automatically open the URL it finds within the database.

With NestJS this is easy because you only have to adjust the unshrink function within the UrlController (src/url/url.controller.ts). Change the previously created function to this implementation:

@Get('/s/:shortenedUrl')
async unshrink(@Res() res, @Param('shortenedUrl') shortenedUrl: string) {
	const url = await this.urlService.find(shortenedUrl)
    return res.redirect(url)
}

Deploy The URL Shortener With Docker

Let's assume you want to deploy the URL shortener in a Docker environment and have to develop a Compose file that can be used to do this.

The first step to do will be to create a new Compose file (docker-compose.prod.yml) and copy the content from the previously created MongoDB file into it. Then add a new service called backend which will be used to install, compile and run the NestJS project within the Docker environment.

As you have developed a custom piece of software there will not be a suitable image on DockerHub and you have to create one from scratch. Because the project is based on NestJS, which is working in a NodeJS environment, you can create a new Dockerfile and use the latest version of node as a base image. Then simply copy the source code, install, build and run the project. The following Dockerfile will be sufficient and should be created in the project root:

FROM node:latest

WORKDIR /usr/src/app
COPY package*.json ./

RUN npm install
COPY . .
RUN npm run build

CMD [ "node", "dist/main.js" ]

This Dockerfile can now be used as the image for the backend service in the Compose file.

version: '3.6'
services:
  db:
    image: mongo
    restart: always
    environment:
      MONGODB_USER: paul
      MONGODB_DATABASE: paulsshortener
      MONGODB_PASS: paulspw
  backend:
    image: paulsshortener
    build:
      context: .
      dockerfile: Dockerfile
    ports:
      - 3001:3000
    depends_on:
      - db
    restart: unless-stopped

Unfortunately, running this Compose file will not work because the URL for the links within our project are hard-coded and will not automatically adjust. Also, the DB hostname and port within the AppModule (app.module.ts) are static.

To fix these problems, you have to add and change something within the AppModule (app.module.ts) and the UrlService (url.service.ts). To be specific, add three new variables that hold the database port, the database URL, and the base URL of the resulting service.

Switch to the AppModule (app.module.ts) and add these two variables above the class definition:

const DB_HOST = process.env.DB_HOST || 'localhost'
const DB_PORT = process.env.DB_PORT || '12345'

Additionally, change the Mongoose part within the imports from the module to correctly use these variables:

MongooseModule.forRoot('mongodb://' + DB_HOST + ':' + DB_PORT + '/paulsshortener')

Then open the UrlService (url.service.ts) and add the baseurl variable above the class definition:

const basepath = process.env.BASE_URL || 'http://localhost:3000/'

Lastly, change the create function to return the complete shortened URL using the newly introduced base variable.

async create(url: string) {
	const createdUrl = new this.urlModel({url: url, shortenedUrl: this.shrink(url)});
	await createdUrl.save();
	return basepath + "s/" + createdUrl.shortenedUrl;
}

Now, that you applied these changes you should adjust your Compose file by adding the available environment variables and adjusting them to your needs:

version: '3.6'
services:
  db:
    image: mongo
    restart: always
    environment:
      MONGODB_USER: paul
      MONGODB_DATABASE: paulsshortener
      MONGODB_PASS: paulspw
  backend:
    image: paulsshortener
    environment:
      - BASE_URL=https://locahost:3001
      - DB_HOST=db
      - DB_PORT=27017
    build:
      context: .
      dockerfile: Dockerfile
    ports:
      - 3001:3000
    depends_on:
      - db
    restart: unless-stopped

Finally, deploy it on your localhost:

docker-compose up -d --build

Your URL shortener is now correctly installed and can be used from your local environment.

Deploy to Production Using Traefik

Now that you have a running Docker Service that can be deployed anywhere you can use it to deploy it in a production environment using Traefik. To do this you will edit the Compose file, add all Traefik-related keywords (labels, networks), and adjust it to your needs.

If you are not familiar with deploying Docker Services using Traefik I can recommend the following tutorials covering basic installation on a single server and a server cluster installation using Docker Swarm:

To deploy your URL shortener using Traefik adjust the labels section. The following part will show and explain how it is done in a single server setup and additionally, there will be a Docker Swarm configuration to download:

version: '3.6'
services:
  db:
    image: mongo
    restart: always
    environment:
      MONGODB_USER: paul
      MONGODB_DATABASE: paulsshortener
      MONGODB_PASS: paulspw
    volumes:
      - db:/data/db
    networks:
      - default
  backend:
    image: paulsshortener
    environment:
      - BASE_URL=https://at0m.de/
      - DB_HOST=db
      - DB_PORT=27017
    build:
      context: .
      dockerfile: Dockerfile
    networks:
      - default
      - traefik-public
    depends_on:
      - db
    restart: unless-stopped
    labels:
      - traefik.enable=true
      - traefik.docker.network=traefik-public
      - traefik.constraint-label=traefik-public
      - traefik.http.routers.pauls-shortener-http.rule=Host(`at0m.de`) || Host(`www.at0m.de`)
      - traefik.http.routers.pauls-shortener-http.entrypoints=http
      - traefik.http.routers.pauls-shortener-http.middlewares=https-redirect
      - traefik.http.routers.pauls-shortener-https.rule=Host(`at0m.de`)
      - traefik.http.routers.pauls-shortener-https.entrypoints=https
      - traefik.http.routers.pauls-shortener-https.tls=true
      - traefik.http.routers.pauls-shortener-https.tls.certresolver=le
      - traefik.http.services.pauls-shortener.loadbalancer.server.port=3000
      - traefik.http.middlewares.redirect-pauls-shortener.redirectregex.regex=^https://www.at0m.de/(.*)
      - traefik.http.middlewares.redirect-pauls-shortener.redirectregex.replacement=https://at0m.de/$${1}
      - traefik.http.middlewares.redirect-pauls-shortener.redirectregex.permanent=true
      - traefik.http.routers.blogs-knulst-https.middlewares=redirect-pauls-shortener
volumes:
  db:

networks:
  traefik-public:
    external: true

Before you can successfully deploy this service with Compose you have to adjust the Host: Use your own BASE_URL and update the traefik configuration (For me, it is at0m.de). After changing both values deploy it with:

docker-compose -f docker-compose.prod.yml up -d

Using a Docker Swarm you can use this Compose file. But, you have to adjust BASE_URL and the placement constraints for the MongoDB service. Then build, push the image to our registry, and deploy it onto your Docker Swarm:

docker-compose -f docker-compose.prod-swarm.yml build
docker-compose -f docker-compose.prod-swarm.yml push
docker stack deploy -c docker-compose.prod-swarm.yml paulsshortener

Additional Adjustments for Live Version

Because you deployed it publicly on your server you should add rate limiting by following this approach: https://docs.nestjs.com/security/rate-limiting.

tl;dr:

Install needed package within the project:

npm i --save @nestjs/throttler

In AppModule (app.module.ts) extend imports-array:

ThrottlerModule.forRoot({
    ttl: 60,
    limit: 10,
})

Then, add ThrottlerGuard to the providers array:

{
    provide: APP_GUARD,
    useClass: ThrottlerGuard,
}

Add the SkipThrottle annotation to the Get /s/ endpoint within the UrlController (url.controller.ts) to ignore rate limiting this specific call:

 @SkipThrottle()

Finally, redeploy your Docker service. Don't forget to rebuild before deploying!

Closing Notes

I hope you enjoyed reading my tutorial and are now able to create, build, and deploy your URL shortener website within a Docker container.

Keep in mind that this is a very basic example without any error handling, and no URL checking. However, this tutorial should be a starting point for developing your version.

If you enjoyed reading this article consider commenting your valuable thoughts in the comments section. I would love to hear your feedback about this URL shortener. Furthermore, if you have any questions about implementing your own version, please jot them down below. I try to answer them if possible. Also, share this article with your friends and colleagues to show them how to use NestJs, MongoDB, and Docker to create their own URL shortener.

Feel free to connect with me on Medium, LinkedIn, and Twitter.