This article discussed the differences between a single volume (/boot + /) and a multi-volume (/boot + / + /var + …). Or to put it another way, one volume with everything on it versus several volumes for different directories. There’s plenty of arguments for both sides. We’re basing our discussion on the following diagram.
Note: I use phrases like “likely” throughout this article because disk space issues have unpredictable consequences. A system may become unstable or possibly just keep humming along. Factor that into the discussion below.
The above diagram shows three scenarios. In each case we’re not including the /boot volume.
- Single Volume = all directories exist on a single volume. If that volume becomes full, the OS will not be able to write to disk and the system will likely become unable to function as normal.
- Multi-volume with /var = all directories exist in a single volume except the /var directory – which has its own volume. If the / volume fills up, the OS will be unstable. If the /var volume fills up, the OS may become unstable (and the applications running on the server may too) as there’s more than just logs in the /var directory, such as any databases or web directories. The only exception would be where the /var volume houses only non-critical data, but that’s not the case on a typical Linux system.
- Multi-volume with /var/lib, /var/log, /var/www = all directories exist on a single volume with the exceptions of the /var/lib, /var/log, /var/www directories which each exist on their own volumes. If /var/lib fills up, any databases (and other things) will be unstable. If /var/log fills up, the system will no longer wrote log files to disk. If /var/www fills up, any web servers will likely be unstable. But the OS has the greatest chance of remaining stable for maintenance – even though the applications (the reason for the server existing) will likely be impacted.
Scenarios compared:
- In scenario 1, the OS (if formatted with EXT4) will have 5% of the allocatable storage reserved for the root user by default. So the system may become unstable if the disk is full, but maintenance should still be possible. In the event of a disk filling up without the sysadmin being aware, the sysadmin can add more disk space and expand the volume to include the new disk space.
- In scenario 2, the OS will still likely become unstable if either of the / or /var volumes fill up. A situation where this scenario is an improvement over scenario 1 might be where the server does not host a database, a web server, or logs (possibly sending the logs to another logging server) and performs some other function not requiring the /var directory at all. It’s hard to think of such a situation.
- In scenario 3, the OS has the greatest chance of remaining stable if the volume being filled is /var/log and not any others. It would mean logs would not be written (which is very bad) but the OS might remain stable regardless.
Space is an issue:
Scenario 1 utilizes disk space most efficiently. Combining this with thin provisioned disks (at the hypervisor level) will give the best disk space utilization outcomes. However, thin provisioned disks perform less well when compared to thick provisioned disks. This scenario also provides the most simple configuration for continued maintenance as there’s only a single volume to manage.
Performance is an issue:
Scenario 3 is best for performance when combined with thick provisioned disks (at the hypervisor level). However, it’s less efficient with disk space utilization. This scenario is more complex to maintain over time as there’s more volumes to consider. It’s not hard to maintain, but less simple than scenario 1.
Combining scenario 1 and 3:
This happens often. A system is built with a single volume and later it’s determined better to split the /something onto a different volume. This could be to house a database, web site, to encrypt a special area of storage, or some other reason. The idea is simple; add a disk of the appropriate type/size and mount it over the top of an existing directory. This is not efficient for disk storage utilization because the data being mounted over still exists.
What are people doing in production?:
The Ubuntu machines on AWS default to a single volume. So one might argue scenario 1 is “incredibly” common. Ubuntu (when installing locally) defaults to a single volume too – although it can be modified at installation time – obviously. Given the choice, I opt for scenario 1 to keep things simple, use disk space most efficiently, and make life easier for the next technician.
