DAOS Quickstart Guide: Running DAOS in a VM

ID 标签 727830
已更新 3/31/2022
版本 Latest
公共

author-image

作者

Introduction

Distributed Asynchronous Object Storage (DAOS) is high-performance storage that pushes the limits of Intel hardware. It's based on Intel® Xeon® Scalable Processors, Intel® Optane™ Persistent Memory and NVMe SSDs.

This simple guide is meant for users that want to run DAOS on a minimal VM system running CentOS 7.9 without any Optane or NVMe storage. The installation steps are similar to a traditional DAOS install, though there are some slight differences with the configuration file and the emulation of storage using tmpfs.

 

RPM Installation

In this section the required RPMs will be installed on each of the nodes based upon their role.  Admin and client nodes require the installation of the daos-client RPM and the server nodes require the installation of the daos-server RPM.

  1. Access to the DAOS Package repository at https://packages.daos.io/v2.0/
sudo wget -O /etc/yum.repos.d/daos-packages.repo https://packages.daos.io/v2.0/CentOS7/packages/x86_64/daos_packages.repo
  1. Perform the additional steps on all nodes:
    sudo yum install -y epel-release

     

  2. We install the DAOS server RPM’s on the server nodes:
    sudo yum install -y daos-server

     

  3. Now install the DAOS client RPMs on the client and admin nodes:
    sudo yum install -y daos-client

 

 

Server Installation

On our server we will perform a Network Scan to find an available provider and note its respective interfaces that are available on our VM instance’s server.yml. 

 

  1. Perform network scan with a provider, in this case we want ethernet so ofi+sockets is our provider in the server.yml
    [root@localhost-server ~]# daos_server network scan
    DAOS Server config loaded from /etc/daos/daos_server.yml
    ---------
    localhost
    ---------
    
        -------------
        NUMA Socket 0
        -------------
    
            Provider    Interfaces     
            --------    ----------     
            ofi+sockets enp0s3, virbr0 
    
    
  2. Update our server.yml file accordingly, updating the fabric iface to match one of the available ones above with the given provider. In addition, specify a scm_mount directory and the size in GB we want to allocate for tmpfs to emulate in SCM. This step is different from a traditional DAOS install which could rely on Intel PMem or NVMe SSD’s.
    [root@localhost-server ~]# vi /etc/daos/daos_server.yml
    

     

    #daos_server.yml 
    name: daos_server
    
    access_points: ['localhost]
    port: 10001
    
    transport_config:
      allow_insecure: true
      client_cert_dir: /etc/daos/certs/clients
      ca_cert: /etc/daos/certs/daosCA.crt
      cert: /etc/daos/certs/server.crt
      key: /etc/daos/certs/server.key
    provider: ofi+sockets
    socket_dir: /var/run/daos_server
    nr_hugepages: 4096
    control_log_mask: DEBUG
    control_log_file: /tmp/daos_server.log
    helper_log_file: /tmp/daos_admin.log
    engines:
    -
      targets: 8
      nr_xs_helpers: 0
      fabric_iface: enp0s3
      fabric_iface_port: 31316
      log_mask: INFO
      log_file: /tmp/daos_engine_0.log
      env_vars:
          - CRT_TIMEOUT=30
      scm_mount: /mnt/daos0
      scm_class: ram
      scm_size: 16 #gb to allocate for tmpfs to emulate SCM
    
    
  3. Create a new directory for our tmpfs mount. Then mount this directory and run df -h to verify the if the mounting is successful.
    [root@localhost-server ~]# mkdir /mnt/daos0/
    [root@localhost-server ~]# sudo mount -t tmpfs -o size=64G tmpfs  /mnt/daos0
    [root@localhost-server ~]# df -h /mnt/daos0
    Filesystem      Size  Used Avail Use% Mounted on
    tmpfs            64G     0   64G   0% /mnt/daos0

 

Client Installation

Verify our server has joined with some simple dmg commands on our agent

  1.  Update our daos_control.yml and our daos_agent.yml files to match our new daos_server.yml in step 2 of the Server Installation.
    #daos_control.yml 
    name: daos_server
    port: 10001
    hostlist: ['localhost']
    
    transport_config:
        allow_insecure: true
        ca_cert: /etc/daos/certs/daosCA.crt
        cert: /etc/daos/certs/admin.crt
        key: /etc/daos/certs/admin.key
    
    #daos_agent.yml 
    name: daos_server
    access_points: ['localhost']
    
    port: 10001
    
    transport_config:
        allow_insecure: true
        ca_cert: /etc/daos/certs/daosCA.crt
        cert: /etc/daos/certs/agent.crt
        key: /etc/daos/certs/agent.key
    log_file: /tmp/daos_agent.log
    
  2. Start our servers with the same commands you’d use in a typical DAOS installation
    [root@localhost ~]# systemctl daemon-reload
    [root@localhost ~]# systemctl start daos_server
    [root@localhost ~]# systemctl status daos_server
    ● daos_server.service - DAOS Server
       Loaded: loaded (/usr/lib/systemd/system/daos_server.service; disabled; vendo>
       Active: active (running) since Sun 2022-01-23 12:07:28 EST; 12min ago
     Main PID: 3149 (daos_server)
        Tasks: 8 (limit: 23548)
       Memory: 12.2M
       CGroup: /system.slice/daos_server.service
               └─3149 /usr/bin/daos_server start
    
  3. Start our agents with the same commands you’d use in a typical DAOS installation
    [root@localhost daos]# sudo systemctl start daos_agent 
    [root@localhost ~]# cat /tmp/daos_agent.log
    agent INFO 2022/01/23 12:20:02 DAOS Agent v2.0.0 (pid 3487) listening on /var/run/daos_agent/daos_agent.sock
    
  4. Now that the server, and client are active, we can perform a storage format, in this case using our tmpfs mount we created earlier
    [root@localhost ~]# df -h /mnt/daos0
    Filesystem      Size  Used Avail Use% Mounted on
    tmpfs            64G     0   64G   0% /mnt/daos0
    [root@localhost ~]# dmg storage format -l localhost.localdomain --force -i
    Format Summary:
      Hosts                 SCM Devices NVMe Devices 
      -----                 ----------- ------------ 
      localhost.localdomain 1           0      
        
    
  5.  Using some commands from the DAOS Tour we can verify our DAOS instance and create a pool!
    [root@localhost ~]# dmg system query -v
    Rank UUID                                 Control Address Fault Domain           State  Reason 
    ---- ----                                 --------------- ------------           -----  ------ 
    0    093ab6ea-e18e-4372-9963-ecb7e66801f8 127.0.0.1:10001 /localhost.localdomain Joined        
    
    [root@localhost ~]# dmg storage query usage 
    Hosts     SCM-Total SCM-Free SCM-Used NVMe-Total NVMe-Free NVMe-Used 
    -----     --------- -------- -------- ---------- --------- --------- 
    localhost 17 GB     17 GB    0 %      0 B        0 B       N/A       
    
    [root@localhost ~]# dmg pool create --size=1G test_pool
    Creating DAOS pool with automatic storage allocation: 1.0 GB total, 6,94 tier ratio
    ^C
    
    [root@localhost ~]# dmg pool query test_pool
    Pool 7e1f5ffb-2f22-40e3-b8fb-c263ea694c3e, ntarget=1, disabled=0, leader=0, version=1
    Pool space info:
    - Target(VOS) count:1
    - Storage tier 0 (SCM):
      Total size: 1.0 GB
      Free: 1.0 GB, min:1.0 GB, max:1.0 GB, mean:1.0 GB
    - Storage tier 1 (NVMe):
      Total size: 0 B
      Free: 0 B, min:0 B, max:0 B, mean:0 B
    Rebuild idle, 0 objs, 0 recs
    

 

"