1. Introduction 1.1. Project/Component Working Name: Memory DR Domain Service 1.2. Name of Document Author/Supplier: bijan.mottahedeh@sun.com 1.3. Date of This Document: August 17, 2008 1.4. Name of Major Document Customer(s)/Consumer(s): 1.4.1. The PAC or CPT you expect to review your project: HS PAC 1.4.2. The ARC(s) you expect to review your project: FWARC 1.4.3. The Director/VP who is "Sponsoring" this project: Jerri-Ann Meyer 1.4.4. The name of your business unit: Solaris Software 1.5. Email Aliases: 1.5.1. Responsible Manager: john.falkenthal@sun.com 1.5.2. Responsible Engineer: bijan.mottahedeh@sun.com 1.5.3. Interest List: ldoms-internal@sun.com 2. Project Summary 2.1. Project Description: This project defines a Memory Dynamic Reconfiguration domain service required to dynamically add/remove memory to/from a guest. 2.2. Risks and Assumptions: None. 3. Business Summary 3.1. Problem Area As customers migrate from legacy SPARC systems to sun4v SPARC-CMT platforms, consolidation, provisioning, and dynamic load balancing become major business drivers. LDoms currently has dynamic CPU capabilities, but no dynamic I/O or memory. Until dynamic IO (VIO DR) and dynamic memory features are deployed, customers cannot exploit the full potential of virtualization. This is directly impacting sales of SPARC-CMT today. Further, Supernova platforms have Memory DR as a specific RR requirement. 3.2. Market/Requester Existing SPARC market, migrating from legacy SPARC to SPARC-CMT w/ LDoms. Scalable Systems Group server product groups within Sun. 3.3. Business Justification SPARC-CMT revenue was $285M in Q2FY08, with LDoms accounting for approximately 1/5th of that. Without full dynamic resource allocation (i.e. CPU/Memory/IO), Sun does not have a fully competitive product for consolidation and virtualization. This will impact future sales of SPARC-CMT. 3.4. Competitive Analysis [ redacted ] 3.5. Opportunity Window/Exposure We are already late to market. Memory DR is needed ASAP. 3.6. How will you know when you are done?: When the LDom manager is able to utilize this service to successfully reconfigure a guest's memory while the guest is running. 4. Technical Description: Memory DR for sun4v platforms is capacity-based and enables addition/removal of an arbitrary amount of memory to/from a running LDom. This is in contrast to the board-based model on sun4u platforms where a system board containing memory is added/removed to/from a running domain. This project creates a new Domain Service that consists of two new messages - a request and response message. The request message allows LDom manager to send a request message to a domain. There are in total five unique types of request messages. The first two message types allow the addition and removal of memory from a domain. Since the removal of memory from a domain can be a long running operation, two additional message types are provided to track a removal operation and to cancel a long running memory remove operation. The fifth message type is used to query the status of memory in the domain. The response message is sent back by the domain for a request message. It includes the result of the attempted operation, status of a ongoing operation or the entire range of memory in the domain, depending on the request sent. Optionally, in response to a request, a human-readable string may also be returned in the response. 4.2 Bug/RFE Number(s) Not Applicable 4.3 In Scope Not Applicable 4.4 Out of Scope Not Applicable 4.5 Interfaces 4.5.1 Overview The new interface is embodied in the format of the messages passed between the control domain to the guest domain OS. For a detailed description of the interface see memory-dr-ds.txt 4.5.2 Imported Interfaces Interface Classification Comments ================================================================= Domain Services Sun Private Domain Services protocol and messages formats FWARC/2006/055 sun4v Machine Sun Private Memory MD nodes defined Description nodes by FWARC/2005/115 4.5.3 Exported Interfaces Interface Classification Comments ================================================================ Memory DR message Sun Private Describes format of formats messages exchanged between LDoms Manager and guest domain OS for reconfiguring memory. Domain Services ID Sun Private service id ("dr-mem") to represent this new service 4.6 Doc Impact None. 4.7 Admin/Config Impact Current behavior: Adding or removing memory to/from an active domain using the LDoms Manager CLI in the control domain currently causes a "delayed configuration" message to be displayed to the user. The actual change does not take place until the affected guest domain is rebooted. New behavior: The requested change will take place immediately and will not require a reboot of the target domain. 4.8 HA Impact None. 4.9 I18N/L10N Not affected. 4.10 Packaging & Delivery This case defines a generic interface and is not tied to a particular architecture. 4.11 Security Impact None. 4.12 Dependencies None. 5. Reference Documents FWARC 2005/633 LDoms: Project Q Logical Domaining Umbrella FWARC 2006/055 Domain Services 6. Resources and Schedule: 6.1. Projected Availability: LDoms 1.2 6.2. Cost of Effort: One staff month 6.3. Cost of Capital Resources: N/A 6.4. Product Approval Committee requested information: 6.4.1. Consolidation Name: LDoms Manager, Solaris 6.4.2. Contributing OpCo/BU/Division Name: Solaris 6.4.3. Type of PAC Review and Approval expected: Fast Track 6.4.4. Project Boundary Conditions: N/A 6.4.5. Is this a necessary project for OEM agreements: No 6.4.6. Notes/Dependencies: None 6.4.7. Target RTI Date/Release: LDoms 1.2 6.4.8. Target Code Design Review Date: TBD 6.4.9. Update approval addition: N/A 6.5. ARC review type: Fast Track (FWARC) 6.6. ARC Exposure: open 7. Prototype Availability: 7.1. Prototype Availability: Q1FY09 7.2. Prototype Cost: Done using existing resources.