Host Becomes disconnected

In my current role I have inherited a largish vSphere 4.1 environment, that has to put it nicely “evolved”.  We have been having host disconnection issues in one of the clusters, coupled with HA Configuration errors.  well today one host disconnected and absolutely refused to reconnect, bizarrely it kept telling us that the username/password combination was incorrect within vCenter.

So we jumped on the ILo and entered the SAME username and combination and low and behold we gained access to the promised land.  Now I know what you are thinking “CapLocks”, but no checked that.

My next troubleshooting step was to restart the management agents on the errant host but again this did not fix the error.

The I had a “lightbulb” moment, as we checked the network settings on the host. I saw that the gateway address was incorrect, it was configured with a correct gateway address but for the wrong subnet.  it was set to the vMotion network not the Management network. reset that and all of a sudden we could rejoin vCenter.

I then checked the rest of the Hosts in the Cluster and found the same basic configuration error, we reset the gateway across the cluster and low and behold now HA works as designed and no more configuration errors.

Moral of this story is “Check the Basics”

I have been a little remiss with my blog recently.

I have been a little bit remiss with my blog recently due to personal issues,  I hope to change that.

These are some of the things that I will be blogging on over the next couple of months.

  • P2V – a 101 and how to move it to an industrial scale.
  • View 5.2 a 101 and then an advanced series

Further and contrary to popular belief, the world does not know what Cloud is,  there are still a lot of gaps in peoples understanding.  I will therefore be doing a series on what is cloud and what are the differing aaS’s out there ( and no that is not a naughty statement – it stands for As A Service 😉 )

 

Rimmergram Blog #10:- Exciting Technology times

As ever, the market in which we’re all focused continues to grow and then converge.  The recent acquisitions by VMware of DynamicOps and Nicira are two key examples.  I’m very lucky to have worked, and be working, with some of the game changing technologies in cloud and virtualization.  Some of my past clients’ companies have been gobbled up by bigger players and some of the others continue to stay ahead of the game by product innovation and good sales and marketing .

Currently, I’m excited to be working with 2 emerging startups that have really unique capabilities and IP that i very compelling, not only to an honorary-geek like me but also to customers and partners alike.

One is focused in cloud and virtualization management and the other is focused on cloud and VDI.  One of the things I love about my role is working with vendors that have totally complementary solutions – clearly as a “me, myself and I” business I can’t have competing solutions in my portfolio J

I’ll start with VMTurbo.  They have seen big success with their Intelligent Workload Management solution, VMTurbo Operations Manager.  In particular, Kendrick Coleman includes VMTurbo Community Edition in his Top 10 Free Utilities for VMware.  The people behind VMTurbo are the ex-SMARTs guys (EMC acquired SMARTS in 2004) and, for me, the real smarts is in their Economic Scheduling Engine.  The way this works is a market-based approach to IT, by and is based on modeling the virtualized IT stack as a service supply chain, where components consume services of other components.  So, for example, excessive demand over supply results in rising prices of that resource. Applications competing over the resource may shift their workloads to alternate resources thus lowering their costs. Using economic solution techniques VMTurbo re-distributes the demand, or increases the supply, as appropriate – really cool stuff.   In summary, VMTurbo automates the decision-making process, prevents, versus predicts, problems, assures application performance at the lowest TCO and simplifies control across any hypervisor or cloud architecture.

As Rachel Chalmers of 451 Research recently wrote, “VMTurbo is firing on all cylinders.”

The other is Nutanix. These guys are really extraordinary, both in terms of product innovation and people – dynamic is an understatement for them!  Like VMTurbo, the founders have already created previous successful companies.  Nutanix counts the people that created the Google File System (GFS) and co-authored distributed database patents among its founders.  Their fundamental offering is ‘SAN-free’ datacenters, but the scale and impact of what they provide is multi-faceted. Nutanix is unique in design approach by taking the storage and compute (server) layers from the traditional datacenter architecture model and converging into a single appliance based on commodity x86 hardware that scales out linearly to thousands of nodes. The benefits delivered combine all the desirable elements of a high-end SAN (shared storage, high capacity, enterprise storage features) with the value of local storage complete with integrated flash technology connected directly to the processors. Nutanix’s definition of “converged” is quite different and is inspired by the approach of Google, Facebook, Yahoo etc. who continue to build enormous datacenter capacity based on commodity x86 servers connected with standard 10Gb Ethernet.

John Abbott of 451 Research recently commented, “The Nutanix Complete Cluster sidesteps many of the costs and complexities of networked storage by taking a new look at the problem.”

Both companies are exhibiting at VMworld San Francisco and Barcelona, so I hope you’ll find the time to stop by and find out more if you’re attending either, or both, events.

 

Things from VMware you really should subscribe too.

VMware are possibly on of the best companies out there on managing So-Me (social media), the amount of information that they provide can be baffling at times, especially now that they have such an impressive array of products. however they do provide a set of very useful email and RSS Feeds, the one that I personally find most useful is the Weekly updated KB, well how else can you keep unto date with so many products.

As I have already said I get a weekly digest in my email every week.  So how can you get this in your email box weekly. Easy just follow these few simple steps.

Navigate to http://blogs.vmware.com/kbdigest/ and click the subscribe link.

Vmware site

you will be forward to another page were you can chose your delivery options

KB Digest

And that is how I keep up to date on the Knowledge Base Articles.

It was time for a Change

As most of you know I have been an independent contractor for a long time, however I have been thinking of a change now for a significant amount of time.  it is with some trepidation that I have stepped back into employed status.  It will be a significant change to have to work for somebody again, but I am sure I am up for the  challenge.

By now you are most likely wondering who has actually managed to obtain my signature, and pull me back out of a life of freelancedom.   Well I have decided after much hand wringing and angst, not about the position but about the change of circumstances it will lead me into.  Yes I will not be earning as much as I was as a free lancer, but there will be  the security of having regular income without worrying where the next money is coming from when a contract ends.  the Gaps between contracts were getting longer and longer, when you actually add up what I earned and took out Holiday time and “looking for contracts” the earnings of a Contractor are not as good as they first seem to be Smile. Once again I digress.

After a long skype conversation with a US friend of whom I have great respect.  I have joined Nexenta as a Sales Engineer, it is a completely different track for me, I have never worked for a vendor and I am really looking forward to the challenge.  I have already been informed that I will be the “lead” for all things Virtual in EMEA.  For those that do not know Nexenta are a Storage Company,  there products are based on ZFS.  now those that really know me, know that I am not as Storage person so I have a very steep learning curve ahead of me but I am really up for that.  I have often stated that is my weakest part of my skill set.

So it looks like I will have to get out the sling backs as Booth babe duty looms for me at VMworld. hopefully San Fran, definitely Barcelona.

Very nice link for downloading Sysinternal files

For those of us who have been in IT for a while doing work in Microsoft environments before will know all about the great tools that the guys at Sysinternals provide,  many will remember being worried when they got bought by the big beast themselves and they became part of Microsoft, but they have continued to provide excellent value add in terms of both costed products under the Wininternals brand and the free tools that go out via the Sysinternals brand.

Sometimes it is a royal pain in the backside getting access to these tool’s as you have to navigate the SysInternals site and individually download each tool from their individual page.

Now they have come to the rescue of us all and provided a new way of downloading them especially if you know the actual tool you need

http://live.sysinternals.com.

The following is the text extract from the “About this site” text file

What is this? This is a file share allowing access to all Sysinternals utilities. We have developed this to test an alternate distribution mechanism for our utilities. This will allow you to run these tools from any computer connected to the Internet without having to navigate to a webpage, download and extract the zip file. If you are unfamiliar with Microsoft Windows Sysinternals, it is highly recommended that you visit the website at http://technet.microsoft.com/sysinternals before using these tools. If you have any questions or comments on this file share, please email syssite@microsoft.com Regards, The Microsoft Windows Sysinternals Team

Thanks to the team for providing this method of getting access to their invaluable tools and Thanks to Twitter for alerting me to it. 

VMware Workstation 8.0 released

This morning VMware released VMware Workstation 8 to the public.

Here’s the link to the What’s New:  Workstation 8 What’s New

and here are the release notes: Workstation 8.0 Release Notes

I was lucky enough to be one of the people who got invited into the VMware beta so I did have some time to testdrive the new features. This allows me to highlight to you here what I personally think of the new Workstation version and the items that are particularly exciting in this new release.
First off, I think this is THE most exciting release of VMware Workstation since version 5. Continue reading “VMware Workstation 8.0 released”

Rimmer’s Round Up of VMworld Las Vegas 2011

VMware is definitely more than just a regular virtualization company and continues along the bleeding edge of innovation.  As Paul Maritz explains in this interview, VMware is growing into an IT Solutions company, focused on 3 key areas:

  • More efficient, automated infrastructure,
  • Renewal of applications – develop new apps to service the Facebook generation and
  • Renewal of end-user computing.

It was the end-user computing ‘angle’ of VMworld that was of most interest to me.  I was entranced Steve Herrod’s Tuesday keynote, always delivered from a business aspect as well as technical, despite him being the CTO.  With around half of the keynote being focused on end-user computing and applications, it demonstrated VMware’s thrust into this arena.

Vittorio Viarengo’s contribution to the keynote was fabulous – both in terms of content but also from an entertainment perspective.  As many have already blogged about, Appblast is really exciting, as is Horizon Mobile and Project Octopus.  It will be interesting, however,  to observe how VMware will deliver all of this cohesively, over time, to truly move beyond the PC-era.

I spoke with many attendees and as always the Hands on Labs (HoL) really were the highlight for many and, as a London VMUG steering committee member, I am really excited to learn that the HoLs will be available for all, not just VMworld attendees, in 2012, learn more here.

A personal highlight for me as a newly “inaugurated” 2011 vExpert, was meeting lots of my fellow vExperts at a program meeting hosted by John Troyer.  I can honestly say I have never encountered such a community surrounding a technology/company.  The opportunity to also meet many of the folks I follow on Twitter was another highlight of VMworld for me.

I spent most of my days working in the Solutions Exchange with one of my clients, Centrix Software. The Solutions Exchange had around 250 exhibitors promoting their wares, ranging from the big and mighty vendors down to new, innovative companies, such as Centrix, in the Innovators Pavilion.  As I blogged prior to VMworld, many companies that have previously participated in this innovator pavilion have subsequently been acquired.

It is unlikely I will be at VMworld Copenhagen this year.  I still feel it’s a mistake to hold the European event so close to the US one, but unfortunately this year there is a major event in the UK, IPExpo being held at exactly the same time as VMworld Copenhagen and it looks like I will be there instead.

If you’re attending Copenhagen, I can guarantee you will experience the same community spirit and “awesome sauce” of a superbly staged event by VMware for its customers – enjoy!

 

 

Being prepared for the unexpected, and the expected

I will start off by saying I tend to consider myself a prepared person for most situations and tend to think in the logic of the flow charts if the result of A is this, then move here, if not move here.  Recently I found myself scrambling after our old SAN that stores our bulk backups took a dump and we lost the controller.

I can’t say this wasn’t expected, but I found myself not prepared with the appropriate plan of action which made for a week and a half of long days and nights with my director on my case to get things back up and running.  While all I needed was a simple storage solution for the bulk backup to push to, there was also a counter part of getting these backups to our tape for offsite storage.  When this all happened there wasn’t a huge amount of panic as the best thing you can do is to stay cool and collected, logically think “what is the best option to move forward with?”

For me this was using our little IOmega ix4 NAS box which allowed me to just create a SMB share and pointed the backups there.  It didn’t quite have the performance of our OLD (IBM DS400) Fiber Channel box, but it did well enough to think, hey this isn’t that bad.  I was able to rerun all the Full backups from the weekend and was on my way to having my temporarily solution in place. This is when part two of our backup process came into place, moving the backup files to tape.  I rudely found that I was not able to get Backup Exec to attach to that SMB share and backup the full files.  While I know I probably could’ve eventually figured out how to get Veritas to see those files it seemed like it wasn’t the best solution for a timeframe of getting a new storage solution in place.

I then went in the direction of creating a iSCSI disk from the ix4 to the backup server running Backup Exec.  This in itself was a small challenge since we’re a FC shop, I have had next to no iSCSI experience, a bit of time on Google was able to help me muddle through the process of setting it up.  I copied the files off of the SMB share, onto a small USB Storage drive, attached the iSCSI drive to the server and copied those files back up.  I thought well, this is good, now I can see the drive through Veritas and I’ve always heard about how good iSCSI is, sweet here is my solution!  This time around, performance on the iSCSI drive just tanked. While I know iSCSI works and the performance is there, or companies like NetApp wouldn’t be in business anymore, my configuration wasn’t up to snuff obviously.  So here is attempt number two in the books, no closer to finding a solid solution to hold off my management from making a rash purchase that I would have to live with for the next five to seven years I started to panic a little.

After a week long process of trying to get this up and running, staying up to check on performance during the backup time window, I was starting to get really frustrated and knew my time was drawing short. The golden rule of IT kept playing over in my head, lose data, lose your job and I was going on a week of no bulk backups.  I was about to go to bed, when my girlfriend reassured me, you’ll find something soon, and that is when the light went on in my head.  Earlier that day looking at our production SAN for how much storage we would need to replace our production san (move our production to backups) I remembered there was a 700GB chunk of storage not being used.  Funny thing was the DS400 had 700GB of storage we were using for backups… a perfect match. There should be no performance issue going onto a more powerful SAN and it’s on the fiber network.  The biggest obstacle would be convince my director to allow me to carve this logical drive out for backups, where he always put his foot down that he wanted backups and productions strictly separated.  That night I sent an email stating all the reasons while we should use this as our temporally solution, he agreed to my idea and off I went.

Right now that is the solution we’re using and it seems to be holding up, at least for the time being.  The moral of the story is to think what equipment you have, the performance of that equipment and how are you prepared for the unexpected, or in my case the expected.  While I thought while we have this Iomega ix4 which I had been using for my R/D lab and it had worked well for that, it couldn’t handle 15-20 concurrent connections all dumping backup files.  If we didn’t have that 700GB on our production storage, I would probably still be in the weeds.  While you can’t always have the equipment onsite to counter any problem, it doesn’t hurt to think about where you are vulnerable or single points of failure and what you might have to do to counter those issues if they ever arise.