jdptechnc wrote:
Things like mass A/V updates, Windows Update downloads/installs, etc are bound to cause performance issues if the typical performance footprint is near a potential bottleneck. Most recently for one of my environments, extremely high storage latency was encountered from the automatic quick scan that is initiated when definations are updated, because every VM was issuing a high number of read OPS at the same time, which caused many of the guest OSes to stop responding on the network briefly because they couldn't access their disks quickly enough. It wasn't a networking issue at all, it was just the storage not being able to keep up.
I had this issue in a previous environment, but my disk activity during peak times is never higher than 50 Mbps per vCenter. Maybe that's the issue, but I do have teamed 1 GB nics for storage on both my NICs so I can't see how that would be an issue. Or perhaps there is an issue and I don't know how to determine any storage related issues. On top of that, it's intermittent. For example, it hasn't happened today (yet) and I'm copying 200 GB of files from one server to the other and there was nary a hiccup during a mass definition update a few minutes ago.
This made sense when one of my VM network vmnics was running at 100 Mbps, but now that it's back to 1 Gbps the sporadic hiccups are perplexing. I also have a backup agent on each machine that runs a snapshot at various times throughout the day. Maybe a snapshot and a definition update at the same time is causing an issue. I'm still investigating. Again, thanks for the input. All of this is extremely valuable.