OpenStack Compendium - The Architecture

In this post we continue our journey in OpenStack with a presentation of the high-level architecture.

OpenStack is not based on a single huge open source project but it has been architected and designed using a modular approach where each component is developed independently by a dedicated project.
This is the big picture of the OpenStack architecture including every module and the relation with the other components:

Because OpenStack is highly configurable, it is possible to create a tailored solution using only a subset of modules in order to cover the specific needs. For this reason, the deployments range from a single server using a single module based on Glance or Cinder to very complex datacentre solutions based on multiple interrelated modules configured in high availability mode. You can also install the entire stack in a single box just for development or evaluation purposes.

Compute: (Project NOVA)
Compute is the virtual machine provisioning and management module. It controls many aspects of the virtualization platform like networking, CPU, storage, memory, creating, controlling, and removing virtual machine instances, security, and access control.
Compute offers a wide range of supported virtualization platform ranging from open source to proprietary

Product	Interface	Description
Kernel Virtual Machine (KVM)	Libvirt	Most popular technology for small scale deployments. Supports advanced operations such as live migration and resize.
Xen	Libvirt	Most popular (along with XCP/XenServer) technology for larger scale and production deployments.
Citrix Xen-Server	XenAPI	Citrix’s commercial version of Xen-based virtualization product. Supports advanced features.
Xen Cloud Platform (XCP)	XenAPI	Citrix’s open source version of XenServer. Supports a subset of XenServer features.
VMware ESX / ESXi / vSphere	vSphere API	VMWare virtualization platform.
User Mode Linux	libvirt	Generally considered a lower performance virtualization option.
Microsoft Hyper-V	Windows Management Instrumentation (WMI)	Microsoft’s hypervisor-based virtualization technology.
QEMU	Libvirt	Provides the basis for most Linux-based virtualization technologies (such as KVM and Virtualbox).
Linux Containers (LXC)	Libvirt	LXC is an operating system-level partitioning technology that allows for running multiple isolated servers (containers) in a single kernel.

The module implements own solutions to manage networking and storage but it can also be integrated with the other modules of the architecture.

Object Storage (Project SWIFT)
Object Storage is a distributed storage system for static data such as generic files, multimedia or virtual machine images. There are multiple layers of redundancy and automatic replication, so a failure in a node doesn't result in data loss, and recovery is automatic. It uses the concepts of Zone and Region in order to write multiple copy of the file in isolated storage (i.e. different servers or different racks up to different datacentres).
It is able to scales horizontally by adding new servers to increase the storage and replication.

Block storage (Project CINDER)
Block storage is essentially a management system of volumes used by OpenStack virtual machines. It supports the snapshots in order to back up data. Snapshots can be restored or used to clone a block storage volume. The volumes can be dynamically attached and detached to\from virtual machine, but multiple machine cannot attach the same volume.

Networking (Project NEUTRON)
OpenStack provides a flexible networking layer for the cloud. It lets the management of multiple networks, control traffic and IP addressing, connect servers and devices to one or more networks.
The Neutron abstracts the management of networking services by using a pluggable backend architecture responsible for interacting with the underlying infrastructure. 
A large number of plug-ins are available and include:

Open vSwitch Plugin
Cisco UCS/Nexus Plugin
Linux Bridge Plugin
Modular Layer 2 Plugin
Nicira Network Virtualization Platform (NVP) Plugin
Ryu OpenFlow Controller Plugin
NEC OpenFlow Plugin
Big Switch Controller Plugin
Cloudbase Hyper-V Plugin
Brocade Neutron Plugin Brocade Neutron Plugin
IBM SDN-VE Plugin

OpenStack networking implements a framework to extend the capability with intrusion detection systems (IDS), load balancing, firewalls and virtual private networks (VPN).

Identity services (Project KEYSTONE)
Keystone provides identity management with authentication and authorizations services using a central repository of users mapped to the OpenStack services they can access.
OpenStack identity integrates with existing backend directory services such as LDAP. It supports several forms of authentication including username and password, tokens and AWS-type logins.

Image services (Project GLANCE)
The OpenStack Image Service manages virtual disk images. The images can be used as a template to get new virtual servers up and running. It can also be used to store and catalogue an unlimited number of backups. Glance supports many different storages and formats.

Image Store	Description
Fileystem	Stores, deletes, and gets images from a filesystem directory or shared drive (e.g., NFS).
HTTP	Read-only image store access to get images using URL.
Swift	Stores, deletes, and gets images from a Swift installation.
S3	Deletes or gets images (but not stores) from Amazon’s S3 service

Disk Format	Description
Raw	Unstructured disk format
VHD	Most common format
VMDK	VMware Format
qcow2	QEMU image format, native format for KVM and QEMU.
VDI	Oracle VM VirtualBox virtual disk image format.
ISO	Archive format for optical disks.
AMI, ARI, AKI	Amazon machine, ramdisk, and kernel images (respectively).

Container Format	Description
OVF	An open standard for distributing one or more virtual machine images. Read more about this standard at http://www.dmtf.org/standards/ovf.
aki, ari, ami	Amazon kernel, ramdisk, or machine image (respectively).
Bare	No container for this image.

Metering and Monitoring services (Project CELIOMETER)
The primary goals of this module are monitoring and metering, but the framework is easily expandable to collect for other needs. It can be used as unique point of contact for billing systems to acquire all of the measurements they need to establish customer billing, across all current OpenStack core components.

Dashboard (Project HORIZON)
The dashboard provides a web-based interface to many of the OpenStack services including Nova, Swift and Keystone.
Horizon is based on a Django module called django-openstack. Django is a high-level Python Web framework to create web application (https://www.djangoproject.com/ ).

Orchestration (Project HEAT)
Heat provides a framework to manage the entire lifecycle of infrastructure and applications within OpenStack clouds. Heat provides a template based orchestration for describing a cloud application by executing appropriate OpenStack API calls to generate running cloud applications.
The templates are based on AWS CloudFormation used for creation of most OpenStack resource types (such as instances, floating ips, volumes, security groups, users, etc), as well as some more advanced functionality such as instance high availability, instance autoscaling, and nested stacks.

This is a high-level presentation of the OpenStack architecture in order to describe the main features and capabilities of each single module. Each module could be further drilled-down to illustrate the architecture and the internal work, but it is not the scope of this post.

Now, it is really the time to put your hands-on!

Read More »

OpenStack Compendium

I'm a great fan of Amazon Web Services, perfect for creating pure cloud architectures without compromises, and I follow with great interest the evolution of Microsoft Azure. Anyway, both are proprietary platforms, but what about open platforms?
The open approach has always been a revolutionary concept in the computer history, it decided the success of the platforms and technologies: the success of DEC with PDP was also due to the publishing of detailed specs about inner workings and interfaces; the IBM and the era of  PC-compatible computers based on an open architetture and non-proprietary components; the use of the C language and the openness of the source code by AT&T; the GNU\Linux and the OpenSource movement; the open approach of the Java architecture with "write once and execute anywhere"…..

Recentlty, I had the opportunity to evaluate this compelling cloud platform so I decided to share with you some notes taken during my investigations.

The OpenStack Project aims to create an open source cloud-computing platform for public and private clouds aimed at scalability without complexity. One of the defining core values behind the project is its embrace of openness with both open standards and open source code.

OpenStack has been released under the Apache 2.0 license. Since that first release it has grown into a large community supported by over 12,000 contributors in nearly 130 countries, and more than 850 companies including Red Hat, Canonical, IBM, AT&T, Cisco, Intel, PayPal, Comcast, CoudWatt (see http://www.openstack.org/foundation/companies/ ).

     13/11/2013 

The platform is open and designed to support a broad range of products, technology and standards. It has no proprietary hardware or software requirements, and it integrates legacy systems and third-party products. It also provides a management API with a compatibility layer with the Amazon Web Service platform and Eucalyptus.

The OpenStack is a collection of open source technologies delivering a massively scalable cloud operating system. It has been architected and designed using a modular approach where each component is developed independently by a dedicated project.
A specific OpenStack implementation may leverage of all components or only a subset according the specific needs. The cloud platform provides a full set of feature required to cover any aspect of a cloud service.

Self-management
The easy provisioning (and deprovisioning) of computing capabilities according the needs automatically, without requiring human interaction with each service provider, allows a formidable way to respond just-in-time to any change required by the business.
The on-demand self-service management includes virtual instances, network, VPN, etc…

Easy VM provisioning
The platform includes a complete images management system with a catalog service for storing and querying virtual disk images. The images can be used as a template with a preconfigured application to get new virtual servers up and running in a very short time.

Metering and reporting for billing
The metering capabilities allow keeping a detailed track of the resource usage in order to adopt a pay-per-use approach based on consumed resources.

Monitoring
An extensive monitoring system allows controlling the infrastructure in order to identify exactly the system usage and carefully plan for future expansion. The on-demand resource provisioning can also be used to cope with peak periods when the service is heavily loaded.

Automation
The entire platform can be fully managed by using a broad set of API. The instance provisioning/deprovisioning, network configuration and monitoring can be fully managed programmatically. This reduce drastically the effort to manage the infrastructure and reliability of the delivery processes.

Interoperability and Portability
The Open Source approach of the architecture facilitates a seamless integration of different platforms and technologies. It provides the support to a wide range of visualization platforms ranging from proprietary software (VMWare ESXi and MS Hyper-V) to open source solution (KVM or Citrix XenServer). Additionally, the support of the project by the best of breed HW vendor like Intel, HP, IBM, Dell and Cisco guarantees a great level compatibility.

The Open Stack is one of the most promising platform for cloud computing due to the open source nature and the wide adoption for the cutting the edge technology providers:
 - IBM: cloud offer, IBM Smart Cloud, based on Open Stack project. Last year IBM aquired Softlayer which is based on a different platform, CloudStack, but currrently it is working on a  compatibility layer called Jumplayer (http://blog.softlayer.com/2014/building-bridge-openstack-api)
 - HP: enterprise-grade public cloud based on OpenStack technology
 - Canonical: release of Ubuntu Open Stack solution for private cloud
 - Red Hat: Red Hat's OpenStack Cloud Infrastructure Partner Network is the world's largest commercial OpenStack ecosystem
 - Rackspace: worldwide leading provider of cloud service based on OpenStack
The first class contributors embracing the project guarantees a stable development of the platform. The top corporate contributors are Red Hat, HP, Rackspace and IBM (http://activity.openstack.org/dash/releases/index.html?data_dir=data/havana ). The Havana release has been developed with the contribution of 933 developers.
The roadmap is very challenging with new features added every release (2 releases yearly at April and October). The community around the project is very active and spread around the world including Italy and France. Many conferences are dedicated to OpenStack (14 in the 2013) with a great participation. More than 3,500 people attended the 2013 OpenStack Summit in Hong Kong.

All the premises are very good, but what the future holds?

Read More »

Renting a flat or buying a cottage

Embrace the cloud computing

The Cloud Computing is a topic that I often endeavour to explain even to IT specialists. According to the definition provided by Wikipedia, <<Cloud computing is the use of computing resources (hardware and software) that are delivered as a service over a network (typically the Internet)>>.
Furthermore, the NIST enriches the definition presenting the 5 characteristics required by cloud services and proposing three service models (Cloud Software as a Service (SaaS), Cloud Platform as a Service (PaaS), Cloud Infrastructure as a Service (IaaS)). Although the above definitions are very precise and well structured, there is evident difficulty in understanding the concepts behind the cloud computing and identify the key factors to carefully assess before embrace a cloud service.

In these situations, when you must communicate complex concepts in a simple way, the use of a simile often comes in handy. The classical approach to IT management compared to the cloud approach may be considered similar to renting a flat in a city against buying a cottage in the countryside.

Using a cloud service is like living in a flat. It is possible to live quietly with tidy and respectful neighbours or face coexistence challanges (instances/tenants using the same HW). It is necessary to deal with shared services such as stairs, balconies and garden (computing, storage and networking) and be sure that the residents comply with building regulation (QOS policy of the provider). Obviously, a building located in the downtown residential area is definitely more reliable than one located in a suburb.

Estimate the Provider Reputation

How long does the provider operate in the business?

What is the provider history?

What are the clients? Case studies and Testimonials?

There will be in the next 5 or 10 years?

At the same time, it is important to look carefully at the security features provided like the alarm system, the armoured door, the surveillance. In the case of an owned house, I could setup all the stuff to guarantee an acceptable security. In a rented apartment, I can only accept the current security systems or negotiate additional security with the flat owner.

Request a Trustworthy Service

What is the Service Level of Agreement (SLA)?

What is the guaranteed uptime? Yearly or monthly?

Are there service credits in case of failure?

What are the certifications and accreditations owned by the provider?

ISO 27001? SAS70? PCI?

Moreover, a flat in the city enables us to benefits from services and opportunities not available or affordable in countryside. For example the proximity to the work place (provide global services with low latency by using CDS), mobility services (access the service from everywhere using Internet and from every device), social services and security services.

Take Advantage of the New Opportunities

What is the full range of services of the cloud provider?

Rethink the current in-house service; do not just move to the cloud.

May I benefit of additional security or availability?

A rented flat lets us easily move to another location (provider change) if our needs change and are no longer satisfied by the current accommodation. Another reason for a forced relocation happens if the owner simply goes bankrupt.

Keep your door open

Adopt service loose coupling principle for provide/api/platform

Are used Open Standard or widely used technologies?

Are there procedures or standard data formats or services interfaces

that could guarantee data, application and service portability?

Are there additional costs for data export?

Rent an apartment in town definitely brings undoubted advantages such as the minimal front-end investment required due to the possibility of changing flat as long as the needs change (Rapid Elasticity). There are no maintenance costs. You can also enjoy common services (Resource Pooling). Living in the city the mobility is simplified by using subways, trains and taxi (Broad network access). Anyways, as long as the needs increase requiring a house of considerable size, a car, a garage, the costs can considerably increase.

Save money redesigning your service according the cloud principles

Take advantages of no upfront-costs.

Take advantage of the elasticity, no spare resource.

Take advantage of the reduced operational costs.

Adopt the KISS approach.

Consider that the cost may be greater than colocation or in-house on large scale.

The Cloud Computing is an interesting opportunity for IT. It unleashes development and operation functions from the chains of the physical equipment. It allows redesigning the service according to the agile principles generally adopted only in the software development (this cloud be a topic for a new post). Moreover, it opens up solutions and service levels not affordable in the classical approach unless of huge investments.
Do not make the mistake by simply considering Cloud Computing as just a new technology. It is a new paradigm in the development of infrastructure and services. Using the estabilished approaches and methodologies in the cloud is worthless and, sometime, even counterproductive.

The three key words of Cloud Computing are speed, elasticity and new opportunities.

Have fun!

Further readings:              

               The Wikipedia Definition of Cloud Computing
               The NIST Definition of Cloud Computing
               Cloud Computing Security Risk Assessment

Read More »

Change, Change, Change,.......

The reason for this blog could not be better explained by the following Charles Darwin's quote:

<<It is not the strongest of the species that survives, nor the most intelligent, but the most responsive to the change.>>

Well, this blog has been created with the desire to understand the big changes happening nowadays in ICT. There is an overwhelming amount of new technologies, methodologies and products that constantly change the scenario. New words like Private\Public\Hybrid Cloud, SaaS, PaaS, IaaS, CDN, Cloud Broker, VPC, Cloudstorming, Elasticity, BYOD, BOYA, Tablet are flooding our daily work today. Avoid being overcome by events and being able to exploit the new opportunities is crucial to success the cloud era.

The old schemas characterizing by the last decade needs to be dramatically reviewed. Some posts in this blog may be very close to provocation, however their purpose is to lead me and the blog's reader to rethink and reassess our preconceptions  about products, technologies and methodologies.

This blog will be also the home of products evaluation, technologies analysis, code examples and best practices on day-by-day activities.

Now I don’t know exactly what else to include in the blog and, considering these assumptions, it could not be differently. Don't hesitate to contact me for comments, suggestions or any contributions.

Enjoy this blog! 

Read More »