Posted by Steve Putre on Wed, Jul 21, 2010 @ 10:02 AM
A big part of our work here is the provisioning and management of a Solaris Container Farm for a large customer. In the course of migrating an older Solaris 9 server to a Branded container, users began reporting that their Oracle 11g utilities (namely, opatch and dbua) were not working inside the container. When invoking them, they would receive the error message,
"Could not reserve enough space for code cache"
This issue was observed only in branded containers, not in Solaris 10 Native containers.
After performing some research and contacting the vendor for support, it was determined that there is a wrokaround for the problem: Add the option "-XX:-UseLargePages" to the JVM invocation.
The permanent fix should be to add Solaris patchid 143357-03 or greater to the global zone. We have not tested this as yet.
Posted by Steve Putre on Fri, Apr 16, 2010 @ 10:57 AM
Solaris Zones are an increasingly popular technology for performing server consolidation in large datacenters. Zones are part and parcel of Solaris 10 and will run on any hardware platform (Sparc or X86) where Solaris 10 is supported. On its own, Zones technology addresses the security and isolation aspects of consolidating multiple applications or workloads on a physical host. Combined with other products and technologies, Zones provides the basis for a complete managed solution. Here are some examples of how the zones concept can be expanded and complemented:
Resource Management: Sure, you can cram 20 or more zones onto a single host, but how to ensure that they all get their fair share of CPU and memory resources? Solaris 10 not only provides free, built-in resource management; but gives you the choice of how to implement it. Want to dedicate CPUs to a workload? Want to limit how much real memory and swap space each zone occupies? Or how about allowing any zone to grab as much CPU and memory as it can but reliquish some of that resource when required by other workloads? All built into Solaris 10.
High Availability: Putting all those eggs into one basket could be risky. Both Solaris Cluster 3.x and Symantec's Veritas Cluster Server (VCS) 5.0 provide full support for monitoring anf failover of zones between hosts in a cluster. Solaris Cluster takes the HA concept one step further, by enabling not only the failover of a zone with all of its workloads, but also a 'virtual host' clustering mode, where the applications are monitored and can be moved between two or more zones located on different hosts in the cluster.
Branded Containers: Solaris 8 and Solaris 9 branded containers allow you to retire old hardware by taking a flash image of the physical host and installing it into a container on a Solaris 10 host. The branded container presents its applications with system call interfaces exactly as Solaris 8 or Solaris 9 would; in fact, because the branded container is installed with an image taken directly from a physical host; all binaries and libraries are carried over. The magic occurs at the system call layer, where Solaris 8 or 9 system calls are translated, executed in the Solaris 10 kernel, and the results sent back to the caller in native format. But branded containers are intended as a means to an end-- enabling migration off old hardware and operating systems while planning for migration to native Solaris 10 is underway. Vendor support for a given brand follows the support schedule for its corresponding operating system. When Solaris 8 enters its EOSL (end of support life) in March 2012, there will no longer be patches or support for the Solaris 8 container brand.
Capacity planning and management: A major advantage of shared / consolidated environments is their ability to make much more efficient use of compute resources such as CPU and memory. To fully exploit this advantage, it is important to have historical capacity data upon which to base decisions regarding capacity. Most data used in Solaris capacity planning originates with the kernel's extended accounting facility. The collection tool's local agent will take samples of this data and store it in a centralized database for analysis and reporting. Major capacity management solutions (BMC Perform/Predict, Teamquest, Sun Ops Center) are all container-aware and can give planners a graphical, unified view of where their environments have excess capacity and where capacity is short.
Posted by Steve Putre on Thu, Sep 17, 2009 @ 09:03 PM
XVMOC adapts to a wide variety of customer environments through the use of proxy hosts and operating modes.
The architecture of XVMOC consists of a single Enterprise Controller which communicates with any number of proxy servers. The proxy servers communicate with the agents on the managed hosts. Proxy software can be co-located on the same host as the EC or can reside on separate hosts, commonly in geographically distant or security-isolated datacenters.
The other architectectural building blocks of the XVMOC system are the agents (one per managed host or Solaris Zone), and the web browser, which connects directly to the EC.
The EC does not communicate directly with agents. The proxy is responsible for all communication with the client agents. All communication among the layers is secure HTTP.
The net result of this architecture is a system which can scale to very large global deployments.
To accomodate customers operating in controlled environments, XVMOC supports three operating modes:
- Fully connected, where patches, packages, and active knowledge are downloaded directly from Sun over the internet and customer inventory data is uploaded to Sun's facilities,
- Semi-disconnected, which is the same as fully connected except that inventory data is not uploaded to Sun,
- Disconnected, where the EC is not connected to the internet and data is loaded into the EC's local database using CDs or DVDs
Posted by Steve Putre on Thu, Sep 17, 2009 @ 09:02 PM
Anyone who has managed the Solaris operating system knows that Solaris patch and update management has historically been a time-consuming, anxiety-provoking, and sometimes frustrating process. What has made Solaris patching painful? Let's dig up some bad memories...
- You had to spend hours reading README files, trying to figure out patch dependencies, special install instructions, and system requirements.
- You needed to download every possible patch from Sunsolve, copy them to every server, and attempt to install all of them on your systems, only to have the systems tell you that a good percentage of them are not applicable either because the corresponding package does not exist or the patch is already up-to-date.
- You had to schedule hours of downtime to patch your systems, and in many cases had to manually bring your systems into single-user mode just to add patches
The key to an efficient patching process is active knowledge. For the past three years, Sun has maintained a database of dependencies for every patch and package released not only by Sun, but also by RedHat Enterprise Linux and Suse SLES. Every patch which is released by these vendors is, within a day or two of its release, vetted and entered into a database which tracks all of the patch's dependents, dependencies, and requirements. The vetting process goes beyond the contents of the README file and reaches down to the level of each binary and library in the patch bundles to determine dependencies, conflicts, and requirements.
So Sun has done our patch research for us! How do we get our hands on this knowledge? Answer: Sun's XVM Ops Center (XVMOC) product. Ops Center has been evolving for a number of years now, starting with Sun's aquisition of Aduva and its OnStage product. This evolved into Sun Update Connection, and then to XVM Ops Center 1.0. XVMOC is now in its 2.1 release, with version 2.5 due to be released in the fall of this year.
Having spent much of my career managing, installing, and upgrading Solaris systems, it is clear that XVMOC 2.1 represents a new emphasis on systems management for Sun. It makes the process of patching and upgrading systems much easier and faster than ever before, by:
- Applying patches to many hosts simultaneously
- Determining in advance which patches need to be applied, downloading only the necessary ones from Sun, and copying only necessary packages to each individual host
- Providing the ability to perform dry-runs to reduce the chances of a problem during the patch process
- Enabling quick and clean rollback of any changes to a system
- Support of Sun-recommended and customer-custom patch baselines
Compared to many past products, the browser-based GUI is polished and easy to navigate. The core functionality of managing and installing patches has been augmented by a long list of additional features and capabilities, particularly:
- Bare-metal provisioning of Solaris, Redhat RHEL, and Suse SLES operating systems on Sun and non-Sun hardware.
- Ability to perform firmware upgrades on Sun servers
- Deep monitoring of Sun hardware, ala SunMC
- Direct communication with Sun ALOM, ILOM, and XSCF service processors
- Ability to provision arbitrary products, packages, and scripts onto Solaris, RHEL, and SLES systems
- Tracking of system capacity parameters such as historical CPU, memory, and filesystem usage
- Management of Sun Logical Domains (LDOMS)
XVMOC can integrate with various Enterprise Systems Management platforms, such as Openview, Netcool, and EMC Smarts through the use of Halcyon's Neuron integration module. System alerts captured by XVMOC can be forwarded to the ESM for action.
In future releases, XVMOC will have the ability to perform full lifecycle management of Solaris Containers. Stay tuned for more information.