~/mongodb/nodejs-mongodb-mongoose-number-open-connections-increasing.md

Nodejs with MongoDB - Number of Opened Connections Keep on Increasing with Mongoose Library

2 min readJune 01, 2022by Gorav Singal#python#django
// TL;DR

Prevent MongoDB connection leaks in Mongoose by properly closing old connections before reconnecting, instead of creating new connections on each disconnect event.

Nodejs with MongoDB - Number of Opened Connections Keep on Increasing with Mongoose Library

Introduction

In one of my Nodejs app, I was using Mongoose to talk to MongoDB. I have used some event handling that whenever connection got disconnected, it tries to re-connect. But, over the period of time, MongoDB started complaining that the number of opened connections are getting very high. It was strange as I was not using much connections.

Old Code

const mongoose = require('mongoose');

class DatabaseController {

    //...some code

    initialize() {
        const db = mongoose.connection;

        db.on('connecting', () => {
            logger.log('info','connecting to MongoDB...');
        });

        db.on('connected', () => {
            logger.log('info','MongoDB connected!');
        });
        db.on("disconnected", () => {
            logger.log('error','MongoDB disconnected!');
        });
        db.once('open', () => {
            logger.log('info','MongoDB connection opened!');
        });
        db.on('reconnected', () => {
            logger.log('info','MongoDB reconnected!');
        });
        db.on("close", () => {
            logger.log('info','MongoDB connection closed!');
        });
        db.on("error", (error) => {
            logger.log('error','MongoDB error!', error);
        });

        const dbURI = "...your-connection";
        return mongoose.connect(dbURI, { 
            useUnifiedTopology: true, 
            useNewUrlParser: true, 
            useCreateIndex: true, 
            useFindAndModify: false, 
            serverSelectionTimeoutMS: 60000
        });
    }
}

Need to Close MongoDB Connections Explicitly

Earlier, I was even re-connecting on disconnect event. So, it was calling a connect request again.

Even when you are stopping your app in a normal way or your app crashes. It will not close your DB connections.

Goto your MongoDB instance, open mongo shell:

> db.serverStatus().connections
{
	"current" : 12,
	"available" : 838859,
	"totalCreated" : 27,
	"active" : 1,
	"exhaustIsMaster" : 0,
	"exhaustHello" : 0,
	"awaitingTopologyChanges" : 0
}

Look at the current number of connections. Now, with this code. You are not closing connections. And, this will result in lot of open connections at DB side.

You might think of why would DB is not closing stale connections. The reason DB administrators don’t do this, is that they don’t have any idea about the lifecycle of a db script. Some db script might take 1-2 hours to fetch results. So, they do not clean the connections.

Its your responsibility to close the connections.

Solution

Lets try to close the connections:

const mongoose = require('mongoose');

class DatabaseController {

    //...some code

    initialize() {
        const db = mongoose.connection;

        db.on('connecting', () => {
            logger.log('info','connecting to MongoDB...');
        });

        db.on('connected', () => {
            logger.log('info','MongoDB connected!');
        });
        db.on("disconnected", () => {
            logger.log('error','MongoDB disconnected!');
        });
        db.once('open', () => {
            logger.log('info','MongoDB connection opened!');
        });
        db.on('reconnected', () => {
            logger.log('info','MongoDB reconnected!');
        });
        db.on("close", () => {
            logger.log('info','MongoDB connection closed!');
        });
        db.on("error", (error) => {
            logger.log('error','MongoDB error!', error);
        });

        // to prevent connection leaks we close connections on any restart
        ['SIGINT', 'exit', 'SIGTERM', 'SIGUSR1', 'SIGUSR2', 'uncaughtException', 'unhandledRejection'].forEach((sig_event) => {
            process.on(sig_event, (err) => {
                logger.log('error', `Disconnecting mongodb for event: ${sig_event}`);
                if (err) {
                    logger.log('error', err);
                }
                mongoose.disconnect();
            });
        });

        const dbURI = "...your-connection";
        return mongoose.connect(dbURI, { 
            useUnifiedTopology: true, 
            useNewUrlParser: true, 
            useCreateIndex: true, 
            useFindAndModify: false, 
            serverSelectionTimeoutMS: 60000
        });
    }
}

Now, we are closing the db connections on app closing events:

  • SIGINT for ctrl+c
  • exit when app is closing

You might also handle two more events:

  • uncaughtException for unhandled exceptions
  • unhandledRejection for unhandled promise rejections

Hope it helps.

$ subscribe --weekly

One deep dive a week, in your inbox

System design, AI engineering, trading systems and interview prep. Tested code, real numbers. Free, and you can unsubscribe any time.

or read it on gyanbyte.substack.com

Share

Related Posts

Django Python - How to Build Docker Image and Run Web-service on Apache with Python 3.9

Django Python - How to Build Docker Image and Run Web-service on Apache with Python 3.9

Introduction So you have a Django project, and want to run it using docker image…

MongoDB with Mongoose — Patterns and Pitfalls

MongoDB with Mongoose — Patterns and Pitfalls

Schema Design Philosophy MongoDB schema design is fundamentally different from…

Python SMTP Email Code - How to Send HTML Email from Python Code with Authentication at SMTP Server

Python SMTP Email Code - How to Send HTML Email from Python Code with Authentication at SMTP Server

Introduction This post has the complete code to send email through smtp server…

Python Build Quality Gates with Pylint and Unittest Coverage Thresholds

Python Build Quality Gates with Pylint and Unittest Coverage Thresholds

Introduction It is very important to introduce few process so that your code and…

Python - How to Implement Timed-Function which gets Timeout After Specified Max Timeout Value

Python - How to Implement Timed-Function which gets Timeout After Specified Max Timeout Value

Introduction We often require to execute in timed manner, i.e. to specify a max…

How to Solve Circular Import Error in Python

How to Solve Circular Import Error in Python

Introduction To give some context, I have two python files. (Both in same folder…

Latest Posts

How to Stay Valuable as a Software Engineer in the AI Era: A Practical Playbook

How to Stay Valuable as a Software Engineer in the AI Era: A Practical Playbook

In the previous article I argued that AI is not killing software engineering…

How to Review AI-Generated Code: A Checklist for Catching What the Model Missed

How to Review AI-Generated Code: A Checklist for Catching What the Model Missed

A few months back I approved a PR that was almost entirely AI-generated. It…

How to Debug AI-Generated Code: Understand First, Fix Last

How to Debug AI-Generated Code: Understand First, Fix Last

In the last article I wrote about reviewing AI code before it goes in. This one…

The Future of Software Engineering: What AI Changes, What Stays, and Where the Jobs Go

The Future of Software Engineering: What AI Changes, What Stays, and Where the Jobs Go

Every few weeks somebody sends me a scary headline — “AI will replace all…

The Transactional Outbox Pattern: Never Lose an Event Again

The Transactional Outbox Pattern: Never Lose an Event Again

This is one of those patterns that looks like nothing until you get burnt once…

Event Sourcing, Explained: Store What Happened, Not Just the Final State

Event Sourcing, Explained: Store What Happened, Not Just the Final State

Here is a question I like to ask in interviews: your bank account shows a…