You are currently viewing Docker Tutorial – Docker Volume Management

Docker Tutorial – Docker Volume Management

For any containers you launch in docker, the data will be deleted once you delete the container. When you launch a new container there will be a read-write layer created for the container to store the data. Once you delete the container this read-write layer along with the data stored will also be deleted. I will demonstrate this with an example.

Docker Containers Without Volumes

Go ahead and run the following command which will create a container named “ubuntu_base” from the ubuntu image. If the image is not available locally on your machine it will be pulled from the repository and the container will be created.

$ docker container run -dt --name ubuntu_base ubuntu

Docker will create a directory under /var/lib/docker/overlay2/<hash-value> for the container to store the data. To get the directory name(hash values) run the inspect command on the container and look out for the “Data” section to get the path.

$ docker inspect ubuntu_base

Connect to the container and create a test file.

$ docker container exec -it ubuntu_base bash
$ touch testing.txt

Now you can check in the data directory the same file will be created.

$ find /var/lib/docker/overlay2 -name testing.txt

Go ahead and delete the container and you will see the data also deleted.

$ docker stop ubuntu_base
$ docker rm ubuntu_base
$ find /var/lib/docker/overlay2 -name testing.txt

Docker Containers With Volumes

To store your data persistent, you should create volumes and attach them to the container. The volumes are completely managed by docker and you have a set of commands to manage its life cycle.

There are two types of docker volumes

  • Anonymous volume
  • Named Volume

Docker Anonymous Volume

For every volume you create there will be a data directory created in the following location. If you delete the container, the data will still remain in the _data directory. The anonymous volume will not have any names instead it will have a random hash value. 

/var/lib/docker/volumes/<volume hash>/_data

There are two ways you can create an anonymous volume. The first is by using the docker create command. All you have to do is run the following command which will create a volume with the random hash value. This random hash value is the name of the volume.

$ docker volume create
02610529d5f642f46e27095c0033f5a00f1d3ed1b6b0b546261ed86a776b61a0

You can navigate to /var/lib/docker/volumes directory and see with the same hash name a directory will be created. So all the data related to this container will be stored under this directory.

/var/lib/docker/volumes/02610529d5f642f46e27095c0033f5a00f1d3ed1b6b0b546261ed86a776b61a0/_data

The second way would be to use the -v flag along with the docker run command which will automatically create the volume.You should provide the destination directory path for the volume to be mounted inside the container.

# Syntax
-v <destination-directory>:<options>

$ docker run -dt --name py_container2 -v /pyscripts/ python

To get the volume details for this container run the docker inspect command on the container and navigate to the “Mounts” section. You can get the hash ID of the volume and source directory in the local file system and the destination directory mounted in the docker container.

$ docker volume inspect py_container2

        "Mounts": [
            {
                "Type": "volume",
                "Name": "658fff1217e94f9dade4d58e6fecebda22a526508a195c36b1266333066de93f",
                "Source": "/var/lib/docker/volumes/658fff1217e94f9dade4d58e6fecebda22a526508a195c36b1266333066de93f/_data",
                "Destination": "/pyscripts",
                "Driver": "local",
                "Mode": "",
                "RW": true,
                "Propagation": ""
            }
        ],

Docker Named Volume

Docker named volume is the same as anonymous volume but the main difference is for named volume you can create the volumes with names instead of generating random hash automatically. Similar to an anonymous volume you can also create a named volume in two ways. Either through docker volume command or on the fly using -v flag along with docker run command. 

Here I am creating a volume for my jupyter notebook container.

$ docker volume create jupyter_notebook
$ docker volume ls
local jupyter_notebook

When using the -v flag you should provide the name for the volume and the destination directory. The destination directory will be created inside the container if not exist.

$ docker container run -dt --name notebook -v jupyter_notebook:/jnotebook/ jupyter/datascience-notebook

By default both the anonymous and named volume will be mounted in read-write mode. If you wish to mount it in read-only mode add “:ro” to the -v flag like below.

$ docker container run -dt --name notebook -v jupyter_notebook:/jnotebook/:ro jupyter/datascience-notebook

NOTE: Try to use named volume always unless there is a requirement to use anonymous volume. Creating a volume with a descriptive name and mapping it to the container will always be easy for us to track which volume is mounted across which containers.

Docker Bind Mounts

Bind mount is different from docker volumes. Docker volumes will create the data directory under “/var/lib/docker/volumes” but bind mount allows you to mount any directory from your host machine to the docker container. Think of you have a python project which you want to test using different versions of python for compatibility. In this case, you can simply mount the project directory to the docker container and test the compatibility. 

To demonstrate this I am using the python:latest docker image.

$ docker pull python:latest

I have a very simple script named int_version.py that will print the interpreter version.

#!/usr/bin/env python3

import sys
print(sys.version)

My host machine runs with python 3.10.4 so when I run the script I get the following output.

3.10.4 (main, Apr  2 2022, 09:04:19) [GCC 11.2.0]

Now I will launch the container and mount the directory where the script is present. Following is the syntax for bind mount.

-v <source-directory>:<destination-directiory>:<options>

-v                    =>  Flag to define the bind mount
<source-directory>    =>  Absoulte directory path from the host
<dest-directory>      =>  Mount point where the source directory to be mounted in the docker container.
<options>             =>  Additional options related to selinux/fs mounts.

To launch a new container with a bind mount run the following command. Replace the source and destination directory according to your use.

$ docker run -dt --name py_container -v /home/nixzie/Documents/docker_mount/:/pyscripts/ python

NOTE: If either the source or destination directory is not available, docker will create it on fly.

Connect to the docker container and you can see the mounted directory. I am running the python script which will give the python interpreter version from the docker container.

$ docker container exec -it py_container bash
$ cd /pyscripts
$ ./int_version.py
3.10.5 (main, Jun 23 2022, 10:42:52) [GCC 10.2.1 20210110]

Inspecting Docker Bind Mount

You can get the mount details like source, destination, mode, etc by inspecting the container.

$ docker container inspect py_container

      "Mounts": [
            {
                "Type": "bind",
                "Source": "/home/nixzie/Documents/docker_mount",
                "Destination": "/pyscripts",
                "Mode": "",
                "RW": true,
                "Propagation": "rprivate"
            }
        ],

If you have the jq utility installed in your machine you can also run the following command to filter out mounts output.

$ docker inspect py_container | jq .[]."Mounts"

[
  {
    "Type": "bind",
    "Source": "/home/karthick/Documents/docker_mount",
    "Destination": "/pyscripts",
    "Mode": "",
    "RW": true,
    "Propagation": "rprivate"
  }
]

Docker Volume “-v” vs “–mount” usage 

We discussed the usage of the -v flag in the previous sections. The -v flag is used to create named or anonymous volumes and bind mounts when launching the container. Alternate to -v flag you can also use –mount flag to create volumes and bind mounts. When using the –mount flag you have to give the volume information in the form of key and value.

For example, if you want to create a bind mount using the –mount flag, you should provide the type as bind. The source will be the local directory and the target will be the directory to be mounted inside the container. 

$ docker run -dt \
  --name python \
  --mount type=bind,source=/home/nixzie/Documents/docker_mount/,target=/pyscript/,readonly \
  Python

Refer to the following link to know more.

https://docs.docker.com/storage/volumes/#choose-the–v-or—mount-flag

Managing The Lifecycle Of Volumes With Docker Volume Command

To manage the lifecycle of the docker volume you should use the “docker volume” command. We have already seen a couple of these commands in the previous sections. In the upcoming section will show you how to use all the docker volume commands.

To get the list of supported options use the –help flag.

$ docker volume --help
or
$ docker volume
Usage:  docker volume COMMAND
Manage volumes
Commands:  create      Create a volume  inspect     Display detailed information on one or more volumes  ls          List volumes  prune       Remove all unused local volumes  rm          Remove one or more volumes
Run 'docker volume COMMAND --help' for more information on a command.

Create New Docker Volume

To create a new volume before launching the container you can run the following command. If you skip specifying the name then it will create a random hash as the volume name. This is the anonymous volume.

$ docker volume create
02610529d5f642f46e27095c0033f5a00f1d3ed1b6b0b546261ed86a776b61a0

If you pass the name, the volume will be created under that name and will be named volume.

$ docker volume create python_test
python_test

Get The List Of Docker Volumes

To get the list of docker volumes run the following command. If you see my command output, the list will be only for named and anonymous volumes and not for bind mounts.

$ docker volume ls 

DRIVER    VOLUME NAME
local     02610529d5f642f46e27095c0033f5a00f1d3ed1b6b0b546261ed86a776b61a0
local     5ee8ace5307b57cc84890cc2ebdd2fbe3cde6d30807dddaad539d46ed8dcb7f2
local     57f7bb765bde9854d023a85600eb5a08a0120fe2ac67ed5795b52f1f3384c320
local     ab1bb405fb4c6dfffe25bac6178dc15732288488dcee44367b001d65b28738e7
local     jupyter_notebook
local     mariadb_data
local     python_test

Inspecting Docker Volumes

You can inspect the volume and get details like created date, mount point, driver details, etc.

$ docker volume inspect <Volume name>

[
    {
        "CreatedAt": "2022-07-04T10:00:46+05:30",
        "Driver": "local",
        "Labels": {},
        "Mountpoint": "/var/lib/docker/volumes/python_test/_data",
        "Name": "python_test",
        "Options": {},
        "Scope": "local"
    }
]

Inspect output does not show which container is mapped to this volume. To find that you can either run the inspect on the container or use filter like below.

$ docker ps -a --filter volume=python_test
CONTAINER ID   IMAGE     COMMAND     CREATED          STATUS                       PORTS     NAMES
cc1c11edc30c   python    "python3"   18 minutes ago   Exited (137) 9 minutes ago             python

Remove & Prune Docker Volumes

If you want to remove a volume first make sure the docker container is stopped otherwise you will get the following error.

Error response from daemon: remove python_test: volume is in use - [cc1c11edc30ccecaf465214e98c4365ab79ff4b09ddff930223e4a5dc87f5ee0]

To remove the volume run the following command.

$ docker volume rm <container name>                   # Single container
$ docker volume rm <container name> <container name>  # Multiple containers

A volume that is not associated with at least one container is called a dangling volume. To get the list of all dangling volumes run the following command.

$ docker volume ls -f dangling=true

DRIVER    VOLUME NAME
local     5ee8ace5307b57cc84890cc2ebdd2fbe3cde6d30807dddaad539d46ed8dcb7f2
local     02610529d5f642f46e27095c0033f5a00f1d3ed1b6b0b546261ed86a776b61a0

I have two dangling volumes that are not associated with any containers. Now to clean this run the “prune” command which will remove all the dangling volumes.

$ docker volume prune 

WARNING! This will remove all local volumes not used by at least one container.
Are you sure you want to continue? [y/N] y
Deleted Volumes:
5ee8ace5307b57cc84890cc2ebdd2fbe3cde6d30807dddaad539d46ed8dcb7f2
02610529d5f642f46e27095c0033f5a00f1d3ed1b6b0b546261ed86a776b61a0

Total reclaimed space: 172.4kB

Conclusion

So far in this article, I have explained what is docker volumes and bind mounts. I have also explained the usage of the docker volume command. You can also create volumes when you create compose file for which a separate article will be created.

This Post Has One Comment

  1. Akila

    Excellent Article

Leave a Reply

eleven + 20 =