Scalable software controller, the next evolution in Intel Rack Scale Design

In our last discussion, I outlined the need for flexibility and scale in the datacenter and how it necessitates the need for flexible hardware architectures like Intel® Rack Scale Design (RSD). RSD architecture provides pooling of physical resources which can be upgraded by building blocks and not complete servers at a time.

Fulfilling the Intel SDI vision:

In this discussion, I will talk about how this fits into Intel’s Software Defined Infrastructure (SDI) vision and most importantly, help scale the cloud. In June, 2016 we released the first version of a standards-based (Redfish) open source software called PODM (PodManager). It provides a RESTful API-based model of all the resources under a POD, i.e., racks, compute blades, SSD pools, and network elements.

RSD-based OEMs are gearing up to add pools of disaggregated hardware as part of their product offering. Enterprise cloud users would like to benefit from this disaggregated model as the hardware can be upgraded as and when workloads scale, number of users increase or component refresh demands arise. From a cloud IAAS integration usage we recognized that:

1. The lifecycle of an RSD-based node will have these phases: discovered, composed, provisioned, released, upgraded (SW or new HW added) and modified (eg. VLANs added, or reduced).

2. Deployment Software used to deploy a new rack may or may not be used when the same rack has updated hardware, for example a Fuel or a TripleO solution.

3. The “compose” API can be invoked by deployment software or a compute scheduler, like Nova

4. If a new switch or new storage device is added, the requirement to configure and add new VLANs or volumes would need to be initiated.

5. An enterprise user may want to provide specialized services from their RSD racks and to provide differentiated scheduling of instances requiring a flavor based API.

6. A cloud provider may want to group composed nodes from different racks based on user accounts and schedule workloads to a group.

7. An OEM (original equipment manufacturer) may have a need to add VLANS prior to provisioning and would requires to maintain the infrastructure states of pooled resources

In order to address these challenges, Intel and its partners introduced Valence, RSD controller layer at the Openstack Summit in Barcelona, 2016.

The Valence Advantage:

Valence makes it easy to install, schedule, monitor, and manage RSD pools, with an API and controller layer. It is a Software Defined Controller for Rack Scale (Figure 1) to manage the hardware lifecycle of disaggregated hardware. Different layers of cloud management services can control the hardware resources in a multi-service environment without compromising performance and statelessness.

Deployment:

Valence can be called by different deployment software to

1. Discover rack resources

2. Compose a server with required capacity and capability

3. Boot up the bladesand other trays of hardware

4. Release the hardware to the disaggregated pool

The Valence database will keep track of resources in different states that can be accessed by plugins’ API layer (figure 2).

Cloud scalability:

1. Plugins to various OpenStack layers allow the cloud orchestration layers like Nova, Ironic and Neutron to request resources through Valence as and when the capacity needs to grow or when the cloud load balancing policies are activated.

2. The need for pre-provisioning of racks of servers ready to be used but not orchestrated into the cloud is eliminated, thus saving power and energy.

Controller service based architecture for scalability:

Each feature in Valence can be created in the form of a service (“controller” in Openstack parlance). So the controller layer, as shown in the code view of Valence in figure 3, currently has a node controller and a flavor controller. We plan to add more controllers and scale each feature separately.

If you happened to be at the OpenStack Summit in Barcelona, you would have seen multiple presentations on Intel® Rack Scale Design along with demos at the Intel booth. Here is a list for your easy reference:

Demo: https://twitter.com/allisonparker/status/791254955126587392

Brownbag 1: https://www.youtube.com/watch?v=_iHbCOFnsT8

Brownbag 2: https://www.youtube.com/watch?v=XBVBcNwNdxU

And if you’re ready for a technical deep dive, you can explore the details of Intel® Rack Scale Valence components on GitHub (https://github.com/openstack/rsc/tree/master/valence), review planned features, and join the IRC meetings.