Version : 1.2 Date : 10/01/2008 Revision History : 1.0 Original specification filed with 2008/401 1.1 Changed group number from 0x4 to 0x207, this updated specification was filed with 2008/592 1.2 Changed function numbers to 0x13b, 0x13c, 0x13d and 0x13e. 2008/592 got revised with this change. 4. Technical Description: 4.1. Details: As the capabilities of MMUs for SPARC processors increase, it is no longer feasible in all cases to implement a TLB with a single associative lookup. UltraSPARC processors now allow for the following inputs into the TLB lookup: * A Partition id (number of bits depends on CPU) * A tag from the VA (up to 51 bits) * A private context number (up to 16 bits) * One or more shared context numbers (up to 16 bits each) Additionally, the TLB must be able to recognize mappings to as many as 8 distinct page sizes. The total number of inputs means that for CPU hardware to find the mapping for a given memory access, multiple TLB probes may be required. With the CPU performing multiple probes, it may be important for software to control either the number or order of the probes: * Controlling the number of probes may improve performance by eliminating unnecessary probes. * Controlling the order of probes may improve performance by searching frequently accessed mappings before searching rarer ones. * Controlling the order of probes can make it possible to have multiple mappings for the same address without ambiguitiy, by indicating which mapping has precedence. The API below provides control over the order and number of probes performed by the MMU, where each probe can be different in the following parameters: * Which context register (private or shared) is used to determine the context number. * The page size of the mapping to probe for. 4.1.1. MD property updates: 4.1.1.1. Default search order: On systems supporting the TLB search API, there is a default search order that is in effect for all MMU accesses (privileged and non-privileged, instruction and data) if the guest does not use the new API. The characteristics of that default search order are indicated by existing properties in the MD node: Name Tag Description ---------------------------------------------------------------------- mmu-#shared-contexts PROP_VAL A 64-bit unsigned integer giving the number of primary and secondary shared context registers supported by this virtual CPU's MMU and the number of shared contexts in the default search order. If not present the default value is assumed to be 0. mmu-page-size-list PROP_VAL A 64-bit unsigned integer treated as a bit field describing the page sizes that may be used on this virtual CPU and page sizes supported in the default search order. Page size encodings are defined according to the sun4v Architecture Specification. A bit N in this field, if set, indicates that sun4v defined page size with encoding N is available for use. The default search order searches every combination of context register and page size specified in these two properties. The order of the default search is not specified. These properties exist merely to provide compatibility for guests that do not use the TLB search API, and have no meaning for a virtual CPU that has changed its search order. 4.1.1.2. New MD properties: The properties below are part of the CPU node. All of these properties are required to be present for any CPU that supports the new API calls described in this specification. None are present in the CPU node unless the CPU supports the calls. Name Tag Description ---------------------------------------------------------------------- mmu-search-#shared-contexts PROP_VAL The number of shared contexts (in addition to the private context) supported in search list entries. The interpretation is equivalent to the mmu-#shared-contexts property; however, the value may be different. mmu-search-page-size-list PROP_VAL The set of page sizes supported in search list entries. The format is equivalent to the mmu-page-size-list property; however, the value may be different. mmu-max-search-order PROP_VAL The maximum number of entries allowed in a search list. All search lists terminate after this many entries, with or without a terminator entry with its enable bit clear. mmu-priv-search-unified PROP_VAL mmu-non-priv-search-unified PROP_VAL Indicates whether the privileged and non-privileged search orders have unified or separate I- and D- search orders. A value of 1 indicates the search order is unified (I- and D- search orders must be the same). A value of 0 indicates that the I- and D- search orders may be different. 4.1.1.3. Notes and caveats for MD properties: The mmu-page-size-list and mmu-search-page-size-list properties are distinct, and may have different values; similarly for mmu-#shared-contexts and mmu-search-#shared-contexts. To understand why, consider the case of a CPU that supports 8 page sizes and 1 shared context (meaning 2 contexts total). To search every context for every page size requires 8*2=16 probes. If mmu-max-search-order is less than 16, the default search order cannot search every valid combination. To guarantee compatibility for guests that don't use (or know about) TLB search orders, the product of the number of sizes in mmu-page-size-list and mmu-#shared-contexts must be selected to be less than mmu-max-search-order. To achieve this, in some cases the default search order may disable page sizes, context registers, or both. 4.1.2. API specification: 4.1.2.1. Search lists for MMU translations and MMU flag : The new API introduces a new data type for specifying MMU search lists. A search list is the real address of an array of two-byte search entries; the size of the array is specified in the mmu-max-search-order property. Each entry specifies a context register and page size for a TLB probe, plus an enable bit. To guarantee a hit for a particular mapping, the mapping must be installed normally in a TSB, or with one of the mapping API calls; additionally there must be an entry in the applicable TLB search order specifying the page size for the mapping and a context register that holds the context number of the mapping. To guarantee a miss on a particular address it is necessary to demap the address fully. If a mapping is valid with the current context registers and TSB contents, it may hit even if the mapping cannot be found using the current applicable search order. The ordering of entries in the searchlist is significant in two ways: * Probes to entries earlier in the search order may perform faster than probes to later entries. * For the non-privileged search list, if the MMU has two mappings for the same address distinguished by the size of the mappings, a page size that occurs earlier in the search order takes precedence over a size later in the search order. Mappings in the MMU must be unique except in the case of non-privileged mappings distinguished by page size. This includes the case of the same VA being mapped differently by two different context registers, even if one of the context registers is not used in the search order. (This is the same restriction documented in section 14.12.2 of the UltraSPARC Architecture 2007 specification). The API calls for configuring a search order allow specifying flags indicating whether the search order is for instruction accesses, data accesses, or both. However, in some cases, there may be a unified search order for both instruction and data accesses. In this case, the appropriate property in the MD (either mmu-priv-search-unified or mmu-non-priv-search-unified) will indicate that separate search lists are not available. The individual bit fields of each search list table entry are laid out as follows: | Enable | reserved | Context | reserved | Size | [15] [14:11] [10:7] [6:3] [2:0] Enable - The Enable field is 1 to indicate a valid search entry. If a search entry has its Enable bit cleared, that entry and all higher-indexed entries are ignored. Context - The Context field is 0 to indicate CONTEXT0 (the private context), 1 to indicate CONTEXT1 (the 1st shared context), and so forth. The value of the field cannot exceed mmu-search-#shared-contexts (see below). If shared contexts are not supported, the field must be 0. The Context field is only supported for non-privileged search lists; privileged search lists must set Context to 0 (indicating only CONTEXT0 is searched). Size - The Size field encodes a standard page size field: PageSize = 1 << (3 * Size + 13) The size must be one of the valid sizes indicated in mmu-search-page-size-list. The MMU Flag argument for each API is the same as defined by core API bindings (FWARC/2005/116, see link to Virtual Machine Specification in section 5 below) but is duplicated here for quick reference. The flags argument applies the API operation to instruction translations if bit 1 is set, and in addition applies the API operation to data translation entries if bit 0 is set. For every API service requiring a flags argument, at least one of bit 0 and/or bit 1 must be set. 4.1.2.2. API function call summary: RK_MMU_SEARCH (0x207) is created for the MMU extension APIs defined below. These are the calls for configuring the new MMU mapping resources: #Name Group# Trap# Func# Major# Minor# #==== ====== ===== ===== ====== ====== MMU_GET_NONPRIV_SEARCH 0x207 0x80 0x13b 1 0 MMU_SET_NONPRIV_SEARCH 0x207 0x80 0x13c 1 0 MMU_GET_PRIV_SEARCH 0x207 0x80 0x13d 1 0 MMU_SET_PRIV_SEARCH 0x207 0x80 0x13e 1 0 4.1.2.3. trap: FAST_TRAP function: MMU_GET_NONPRIV_SEARCH arg0: searchlist arg1: flags ret0: status The flags argument indicates whether to get the search order for instruction or data accesses. Only one type of access can be gotten with a single call. Fill in the provided search list with the current search list for the calling virtual CPU, as set by the MMU_SET_NONPRIV_SEARCH call. Errors: EINVAL flags is 0, or indicates both I- and D- accesses. ENORADDR The search list is not a valid real address. 4.1.2.4. trap: FAST_TRAP function: MMU_SET_NONPRIV_SEARCH arg0: searchlist arg1: flags ret0: status The flags argument indicates whether the search order applies to instruction or data accesses, or both. The encoding is the same as for other MMU calls, such as MMU_MAP_ADDR. If the value of mmu-non-priv-search-unified is 1, the flags parameter must specify both I- and D- searches together. Set the calling virtual CPU's non-privileged search list. This search list will be used for both instruction and data lookups performed in non-privileged mode, or when using any of the "AS_IF_USER" ASIs in privileged mode. (N.B. the search list applies to non-privileged TLB lookups, regardless of the context to be searched. This is different from TSB lookup, which searches based on the context number). Errors: EINVAL flags is 0 EINVAL mmu-non-priv-search-unified is 1, and flags does not specify both I- and D- accesses. EINVAL A page size in the search list isn't valid. EINVAL A context register number in the search list isn't valid. ENORADDR The search list is not a valid real address. 4.1.2.5. trap: FAST_TRAP function: MMU_GET_PRIV_SEARCH arg0: searchlist arg1: flags ret0: status The flags argument indicates whether to get the search order for instruction or data accesses. Only one type of access can be gotten with a single call. Fill in the provided search list with the calling virtual CPU's current search list for privileged translations, as set by the MMU_SET_PRIV_SEARCH call. Errors: EINVAL flags is 0, or indicates both I- and D- accesses. ENORADDR The search list is not a valid real address. 4.1.2.6. trap: FAST_TRAP function: MMU_SET_PRIV_SEARCH arg0: searchlist arg1: flags ret0: status The flags argument indicates whether the search order applies to instruction or data accesses, or both. The encoding is the same as for other MMU calls, such as MMU_MAP_ADDR. If the value of mmu-priv-search-unified is 1, the flags parameter must specify both I- and D- searches together. Set the calling virtual CPU's search list for privileged translations. Errors: EINVAL flags is 0 EINVAL mmu-priv-search-unified is 1, and flags does not specify both I- and D- accesses. EINVAL A page size in sizelist isn't valid. EINVAL A Context field in sizelist is not 0. ENORADDR The search list is not a valid real address. 4.2. Bug/RFE Number(s): N/A 4.3. In Scope: N/A 4.4. Out of Scope: N/A 4.5. Interfaces: 4.5.1. Imported Interfaces: Interface Classification Comments ==================================================================== Machine Description (MD) Sun Private Defined by FWARC/2005/115 data structures sun4v core API Sun Private Core APIs defined by FWARC/2005/116 4.5.2. Exported Interfaces: Interface Classification Comments ==================================================================== mmu-search-#size-bits Committed MD property mmu-search-#context-bits Committed MD property mmu-search-page-size-list Committed MD property mmu-max-search-order Committed MD property mmu-non-priv-search-unified Committed MD property mmu-priv-search-unified Committed MD property RK_MMU_SEARCH Committed API group 0x207, v1.0 MMU_GET_NONPRIV_SEARCH Committed API call MMU_SET_NONPRIV_SEARCH Committed API call MMU_GET_PRIV_SEARCH Committed API call MMU_SET_PRIV_SEARCH Committed API call 4.6. Doc Impact: The UltraSPARC Virtual Machine Specification must be updated to reflect this addition. 4.7. Admin/Config Impact: No impact. 4.8. HA Impact: No impact. 4.9. I18N/L10N Impact: No impact. 4.10. Packaging & Delivery: No impact. 4.11. Security Impact: No impact. 4.12. Dependencies: This interface depends on underlying hardware that provides the capability. The API and associated properties will not be present on platforms that don't support the capability. 5. Reference Documents: UltraSPARC Architecture 2007 Specification http://opensparc-t2.sunsource.net/specs/UA2007-current-draft-HP-EXT.pdf UltraSPARC Virtual Machine Specification http://opensparc-t1.sunsource.net/specs/Hypervisor-api-current-draft.pdf