[MongoDB]: Linux ulimit Settings

How do we review and set resource limits :

Here’s the command to see unlits is below

[root@dbversitydotcom oracle]# ulimit -a
core file size (blocks, -c) 0
data seg size (kbytes, -d) unlimited
scheduling priority (-e) 0
file size (blocks, -f) unlimited
pending signals (-i) 127368
max locked memory (kbytes, -l) 64
max memory size (kbytes, -m) unlimited
open files (-n) 65536
pipe size (512 bytes, -p) 8
POSIX message queues (bytes, -q) 819200
real-time priority (-r) 0
stack size (kbytes, -s) 10240
cpu time (seconds, -t) unlimited
max user processes (-u) 16384
virtual memory (kbytes, -v) unlimited
file locks (-x) unlimited
[root@dbversitydotcom oracle]#

You can change ulimit settings by issuing a command in the following form:

ulimit -n <value>

There are both “hard” and the “soft” ulimit“s that affect MongoDB’s performance. The “hard” “ulimit refers to the maximum number of processes that a user can have active at any time. This is the ceiling: no non-root process can increase the “hard” ulimit. In contrast, the “soft” ulimit is the limit that is actually enforced for a session or process, but any process can increase it up to “hard” ulimit maximum.

A low “soft” ulimit can cause can’t create new thread, closing connection errors if the number of connections grows too high. For this reason, it is extremely important to set both ulimit values to the recommended values.

ulimit will modify both “hard” and “soft” values unless the -H or -S modifiers are specified when modifying limit values.
After changing the ulimit settings, you must restart the process to take advantage of the modified settings.
Recommended ulimit Settings :
=======================

Every deployment may have unique requirements and settings;
however, the following thresholds and settings are particularly important for mongod and mongos deployments:

-f (file size): unlimited
-t (cpu time): unlimited
-v (virtual memory): unlimited [1]
-n (open files): 64000
-m (memory size): unlimited [1] [2]
-u (processes/threads): 64000
Always remember to restart your mongod and mongos instances after changing the ulimit settings to ensure that the changes take effect.
Purpose of UNLIT settings :
======================

Operating systems like Unix, including Linux and OS X, provide ways to limit and control the usage of system resources such as threads, files, and network connections on a per-process and per-user basis.
These “ulimits” prevent single users from using too many system resources.

Sometimes, these limits have low default values that can cause a number of issues in the course of normal MongoDB operation.
NOTE : Red Hat Enterprise Linux and CentOS place a max process limitation of 1024 which overrides ulimit settings.
Create a file named /etc/security/limits.d/99-mongodb-nproc.conf with new soft nproc and hard nproc values to increase the process limit.

See below /etc/security/limits.d/90-nproc.conf file as an example.

[root@dbversitydotcom oracle]# cat /etc/security/limits.d/90-nproc.conf
# Default limit for number of user’s processes to prevent
# accidental fork bombs.
# See rhbz #432903 for reasoning.

* soft nproc 1024
root soft nproc unlimited
[root@dbversitydotcom oracle]#

Resource Utilization :
=================

mongod and mongos each use threads and file descriptors to track connections and manage internal operations.
Use these figures in combination with the actual information about your deployment and its use to determine ideal ulimit settings.

Generally, all mongod and mongos instances:

track each incoming connection with a file descriptor and a thread.
track each internal thread or pthread as a system process.

mongod

1 file descriptor for each data file in use by the mongod instance.
1 file descriptor for each journal file used by the mongod instance when storage.journal.enabled is true.

In replica sets, each mongod maintains a connection to all other members of the set.
mongod uses background threads for a number of internal processes, including TTL collections, replication, and replica set health checks, which may require a small number of additional resources.

mongos

In addition to the threads and file descriptors for client connections, mongos must maintain connects to all config servers and all shards, which includes all members of all replica sets.
For mongos, consider the following behaviors:

mongos instances maintain a connection pool to each shard so that the mongos can reuse connections and quickly fulfill requests without needing to create new connections.
You can limit the number of incoming connections using the maxIncomingConnections run-time option.

By restricting the number of incoming connections you can prevent a cascade effect where the mongos creates too many connections on the mongod instances.
Linux distributions using Upstart :
===========================

Linux distributions using systemd :
———————————-

For Linux distributions that use systemd, you can specify limits within the [Service] sections of service scripts if you start mongod and/or mongos instances as systemd services. You can do this by using resource limit directives.

Specify the Recommended ulimit Settings, as in the following example:

[Service]
# Other directives omitted
# (file size)
LimitFSIZE=infinity
# (cpu time)
LimitCPU=infinity
# (virtual memory size)
LimitAS=infinity
# (open files)
LimitNOFILE=64000
# (processes/threads)
LimitNPROC=64000
Each systemd limit directive sets both the “hard” and “soft” limits to the value specified.

After after changing limit stanzas, ensure that the changes take effect by restarting the application services, using the following form:

systemctl restart <service name>

/proc File System :
———————–
NOTE :This section applies only to Linux operating systems.

The /proc file-system stores the per-process limits in the file system object located at /proc/<pid>/limits, where <pid> is the process’s PID or process identifier. You can use the following bash function to return the content of the limits object for a process or processes with a given name:

return-limits(){

for process in $@; do
process_pids=`ps -C $process -o pid –no-headers | cut -d ” ” -f 2`

if [ -z $@ ]; then
echo “[no $process running]”
else
for pid in $process_pids; do
echo “[$process #$pid — limits]”
cat /proc/$pid/limits
done
fi

done

}

You can copy and paste this function into a current shell session or load it as part of a script. Call the function with one the following invocations:

return-limits mongod
return-limits mongos
return-limits mongod mongos

Ask Question