Blog

All Blog Posts for Openstack


  • Openstack NFS Backend Causing Total Hangs

    Openstack NFS Backend Causing Total Hangs

    I'm not a big NFS fan ever since I worked as a Linux/Unix administrator way back in the good old days. Sometimes when the NFS server hung, lost network connectivity or something else happened all clients that had mounts from the NFS completely blocked waiting for it to come back up, because it is so deep in the kernel. Аll commands, even "ls", froze and the only cure was forcibly rebooting them to get them back online. Neat, eh?When NFS v4.1 emerged, back in 2010 hopes were that it will fix everything. I was a bit sceptical but decided to give it a shot and true, many new fixes in the protocol and implementation were made that enhanced the stability. Some of them were: blocking locks that allow client to ping the server if the lock is released, not only wait for notifications, timeout for server unavailable, parallel access. From what I saw, I couldn’t really break it beyond repair.As time went by Openstack offered the option to have NFS as a storage backend. We decided to use it for one deployment where we saw this technology as appropriate, because we didn't need highly available storage with replication that occupies twice the space, but we needed Cinder volumes to get mounted across the hypervisors. I had a feeling that something could go wrong while making the installation, because I remembered all those nights rebooting servers from ILo / IMPI.

    READ MORE
  • Backing up your virtual machines in Openstack

    Backing up your virtual machines in Openstack

    Backup is an essential part of the IT infrastructure management. Having HA solutions, RAIDs etc. doesn't free you from the need of backup. In case of a human error all those techniques will not save you, only the backup will. However as the saying goes "Your backups are only as good as your restores", so we have to think about regularly checking our backups for consistency.In Openstack it's highly recommended to use Cinder as the main storage provider. Cinder gives you the possibility to create block volumes and attach them to your virtual machines. The best practice is that you keep all your application data onto volumes and not on the instance disk, this disk should be used for the operation system files only, that come from the OS image ( of course packages installed from repositories will also go there) . In this article we will show you more reasons to do so.What you would typically want from a good backup solution is: online backup possibility, easy restores, consistency, easy management, to use as less space as possible.Although it's possible to have a traditional backup solution installed on every virtual machine, Openstack offers us other options to backup our data using snapshotting. The downside is that you can't have an "incremental" snapshot copy yet, you have to store the full size of your snapshots every time you backup. However the simplicity of backups and more importantly restores is far greater than supporting a "in-VM" backup solution that supports incremental backups. 

    READ MORE
  • Monitoring Openstack Part 2

    Monitoring Openstack Part 2

    In my previous blogpost I was discussing how to monitor RabbitMQ as a centralized message Q of Openstack. Well, that's quite important but the end goal of having cloud are the instance on top of the machine. Most of you and especially the infrastructure guys who dig into monitoring will know what are the most important components to look over.The reason to monitor is to have reasonable planning which is probably the drill in cloud environment where you have spawnlarge number of virtual machines of containers. On the other hand having the data in one glance is very easy to increase the reliability, uptime plan better your architecture and identify the bottlenecks of your setup.

    READ MORE
  • Monitoring Openstack Part 1

    Monitoring Openstack Part 1

    Last year we focused on the Openstack technology and the projects behind it. We decided to stress on it and move our scope in that direction because of the rich features and flexibility that it provides. But as we know great power comes with great responsibility.

    READ MORE
  • Configuring multiple block storage backends in OpenStack Cinder

    Configuring multiple block storage backends in OpenStack Cinder

    If you're an administrator of virtualized environments you have definetly ran into IO performance issues. IO is the first bottleneck that one hits. Luckily persistant storage has evolved troughout the years and lately we see the high performant SSDs at a reasonable price, still far higher to allow organizations to fully migrate to SSD. The hybrid environments become more and more popular as they combine the low costs for traditional HDD with the high performant Solid state drives. One of the key features of the Cinder storage back- end component of Openstack is the flexibility that allows us to have more than one storage backend on our storage node. This gives us the flexibility to diferentiate the IO heavy applications from the more compute-oriented ones that are more heavy on cpu usage. Typical example is to configure a database VM to run from a SSD drive and the application server cluster to be on a normal storage that is heavily read only during startup. Here is how to achive that with OpenStack and Cinder.

    READ MORE