Upgrading vRealize Log Insight to Version 8.12: A Journey with Lessons Learned
As a home lab enthusiast, I use vRealize Log Insight in my homelab for collecting logs from my vCenter and ESXi hosts, and as a syslog server. However, my version of vRealize Log Insight had been stuck on version 8.6 for some time, and with the knowledge that version 8.12 is the latest and greatest, I knew it was time to upgrade. But, as I soon found out, upgrading from an older version to the latest one can be a bit of a challenge. In this blog post, I will share my experience and the lessons I learned during the upgrade process.
Issues with Internal Certificates
One of the main reasons I wanted to upgrade was because of the internal certificates expiring on April 30th, 2023. As mentioned in this KB article, the cert issue is present in all versions prior to 8.12. When checking the release notes of version 8.12, the supported upgrade path is from 8.10.x. This had me check the release notes for version 8.10.2, which shows that the supported upgrade path is from 8.8.x. Digging down to the release notes for version 8.8.2 finally revealed a supported way to upgrade from my 8.6.0 installation. So, my upgrade path was going to be:
8.6.0 -> 8.8.2 -> 8.10.2 -> 8.12
Upgrade Failures and Custom Certificates
I started my upgrade process by taking a snapshot of the VM and downloading the bits needed from Customer Connect. However, my first upgrade failed. I did some quick troubleshooting and suspected that the custom certificate might be the reason. I noticed that the cert in the log message was the internal self-signed certificate and not my own custom one. Digging further, I tried to curl the URL from the vRLI VM itself and found that it didn’t trust my custom certificate.
Adding the Root CA to the VMs Trust Store
To resolve this issue, I needed to add my custom Root CA to the VMs trust store. However, when trying to save the Root CA cert, I got a message stating that it couldn’t be saved due to a full disk. So, after cleaning out some space on the root partition, I added the Root CA cert and tried again. This time, the upgrade succeeded.
Upgrade from the Command Line
However, my next attempt failed again, and this time the log message indicated that there was a missing 16KB on the root filesystem! So, after cleaning up some more space on the root partition, I retried the upgrade from the command line. This time, it finally went through. And the Cluster status page shows the updated version.
New UI Layout
After the successful upgrade to 8.8.2, the next step was upgrading to 8.10.2. I tried the upgrade through the UI once more, and this time I had no issues. After letting the instance rest for about an hour, I continued with the final step, upgrading to 8.12. Unfortunately, this did not succeed on the first try. When checking the update log, I found a message with an error on the SSL certificate.
Removing Custom SSL Certificate
I also checked the disk space, as that had been the cause on previous upgrades, but this time that seemed fine. I decided to remove my custom SSL cert by reverting the SSL config to the default as the release notes mention some considerations around custom certs. After restarting my browser session, I confirmed that the SSL cert was now using the VMware self-signed, so I tried the upgrade again through the CLI to fetch any error messages. And score! The vRLI is now Aria Operations for Logs, and we are running on version 8.12.
Lessons Learned
In conclusion, upgrading from an older version of vRealize Log Insight to the latest version 8.12 can be a bit of a challenge. Here are some lessons I learned during the process:
1. Custom certificates can cause issues with upgrades.
2. Disk space is crucial when upgrading, especially when adding custom certificates.
3. The CLI can be a useful tool for troubleshooting and resolving issues during the upgrade process.
4. The release notes are essential for understanding the supported upgrade paths and identifying potential issues before they become major problems.
5. Adding the Root CA to the VMs trust store is necessary when using custom certificates.
6. Cleaning up disk space before upgrading can help prevent issues related to disk space.
7. The new UI layout in version 8.10.2 is a welcome change and makes navigation easier.
8. Custom certificate considerations should be taken into account when upgrading.
In summary, upgrading from an older version of vRealize Log Insight to the latest version 8.12 requires careful planning and attention to detail. By understanding the potential issues that can arise and taking the necessary precautions, you can ensure a successful upgrade process.