/ appendices / VK_KHR_external_memory.txt
VK_KHR_external_memory.txt
1 // Copyright (c) 2016-2019 Khronos Group. This work is licensed under a 2 // Creative Commons Attribution 4.0 International License; see 3 // http://creativecommons.org/licenses/by/4.0/ 4 5 include::meta/VK_KHR_external_memory.txt[] 6 7 *Last Modified Date*:: 8 2016-10-20 9 *IP Status*:: 10 No known IP claims. 11 *Interactions and External Dependencies*:: 12 - Interacts with `<<VK_KHR_dedicated_allocation>>`. 13 - Interacts with `<<VK_NV_dedicated_allocation>>`. 14 - Promoted to Vulkan 1.1 Core 15 *Contributors*:: 16 - Jason Ekstrand, Intel 17 - Ian Elliot, Google 18 - Jesse Hall, Google 19 - Tobias Hector, Imagination Technologies 20 - James Jones, NVIDIA 21 - Jeff Juliano, NVIDIA 22 - Matthew Netsch, Qualcomm Technologies, Inc. 23 - Daniel Rakos, AMD 24 - Carsten Rohde, NVIDIA 25 - Ray Smith, ARM 26 - Chad Versace, Google 27 28 An application may wish to reference device memory in multiple Vulkan 29 logical devices or instances, in multiple processes, and/or in multiple 30 APIs. 31 This extension enables an application to export non-Vulkan handles from 32 Vulkan memory objects such that the underlying resources can be referenced 33 outside the scope of the Vulkan logical device that created them. 34 35 === New Object Types 36 37 None. 38 39 === New Enum Constants 40 41 * ename:VK_STRUCTURE_TYPE_EXTERNAL_MEMORY_BUFFER_CREATE_INFO_KHR 42 * ename:VK_STRUCTURE_TYPE_EXTERNAL_MEMORY_IMAGE_CREATE_INFO_KHR 43 * ename:VK_STRUCTURE_TYPE_EXPORT_MEMORY_ALLOCATE_INFO_KHR 44 * ename:VK_QUEUE_FAMILY_EXTERNAL_KHR 45 * ename:VK_ERROR_INVALID_EXTERNAL_HANDLE_KHR 46 47 === New Enums 48 49 None. 50 51 === New Structs 52 53 * slink:VkExternalMemoryImageCreateInfoKHR 54 * slink:VkExternalMemoryBufferCreateInfoKHR 55 * slink:VkExportMemoryAllocateInfoKHR 56 57 === New Functions 58 59 None. 60 61 === Promotion to Vulkan 1.1 62 63 All functionality in this extension is included in core Vulkan 1.1, with the 64 KHR suffix omitted. 65 The original type, enum and command names are still available as aliases of 66 the core functionality. 67 68 === Issues 69 70 1) How do applications correlate two physical devices across process or 71 Vulkan instance boundaries? 72 73 *RESOLVED*: New device ID fields have been introduced by 74 `<<VK_KHR_external_memory_capabilities>>`. 75 These fields, combined with the existing 76 slink:VkPhysicalDeviceProperties::pname:driverVersion field can be used to 77 identify compatible devices across processes, drivers, and APIs. 78 slink:VkPhysicalDeviceProperties::pname:pipelineCacheUUID is not sufficient 79 for this purpose because despite its description in the specification, it 80 need only identify a unique pipeline cache format in practice. 81 Multiple devices may be able to use the same pipeline cache data, and hence 82 it would be desirable for all of them to have the same pipeline cache UUID. 83 However, only the same concrete physical device can be used when sharing 84 memory, so an actual unique device ID was introduced. 85 Further, the pipeline cache UUID was specific to Vulkan, but correlation 86 with other, non-extensible APIs is required to enable interoperation with 87 those APIs. 88 89 2) If memory objects are shared between processes and APIs, is this 90 considered aliasing according to the rules outlined in the 91 <<resources-memory-aliasing,Memory Aliasing>> section? 92 93 *RESOLVED*: Yes. 94 Applications must take care to obey all restrictions imposed on aliased 95 resources when using memory across multiple Vulkan instances or other APIs. 96 97 3) Are new image layouts or metadata required to specify image layouts and 98 layout transitions compatible with non-Vulkan APIs, or with other instances 99 of the same Vulkan driver? 100 101 *RESOLVED*: Separate instances of the same Vulkan driver running on the same 102 GPU should have identical internal layout semantics, so applications have 103 the tools they need to ensure views of images are consistent between the two 104 instances. 105 Other APIs will fall into two categories: Those that are Vulkan- compatible, 106 and those that are Vulkan-incompatible. 107 Vulkan-incompatible APIs will require the image to be in the GENERAL layout 108 whenever they are accessing them. 109 110 Note this does not attempt to address cross-device transitions, nor 111 transitions to engines on the same device which are not visible within the 112 Vulkan API. 113 Both of these are beyond the scope of this extension. 114 115 4) Is a new barrier flag or operation of some type needed to prepare 116 external memory for handoff to another Vulkan instance or API and/or receive 117 it from another instance or API? 118 119 *RESOLVED*: Yes. 120 Some implementations need to perform additional cache management when 121 transitioning memory between address spaces, and other APIs, instances, or 122 processes may operate in a separate address space. 123 Options for defining this transition include: 124 125 * A new structure that can be added to the pname:pNext list in 126 slink:VkMemoryBarrier, slink:VkBufferMemoryBarrier, and 127 slink:VkImageMemoryBarrier. 128 * A new bit in tlink:VkAccessFlags that can be set to indicate an 129 "`external`" access. 130 * A new bit in tlink:VkDependencyFlags 131 * A new special queue family that represents an "`external`" queue. 132 133 A new structure has the advantage that the type of external transition can 134 be described in as much detail as necessary. 135 However, there is not currently a known need for anything beyond 136 differentiating external vs. 137 internal accesses, so this is likely an over-engineered solution. 138 The access flag bit has the advantage that it can be applied at buffer, 139 image, or global granularity, and semantically it maps pretty well to the 140 operation being described. 141 Additionally, the API already includes ename:VK_ACCESS_MEMORY_READ_BIT and 142 ename:VK_ACCESS_MEMORY_WRITE_BIT which appear to be intended for this 143 purpose. 144 However, there is no obvious pipeline stage that would correspond to an 145 external access, and therefore no clear way to use 146 ename:VK_ACCESS_MEMORY_READ_BIT or ename:VK_ACCESS_MEMORY_WRITE_BIT. 147 tlink:VkDependencyFlags and tlink:VkPipelineStageFlags operate at command 148 granularity rather than image or buffer granularity, which would make an 149 entire pipeline barrier an internal->external or external->internal barrier. 150 This may not be a problem in practice, but seems like the wrong scope. 151 Another downside of tlink:VkDependencyFlags is that it lacks inherent 152 directionality: There are not ptext:src and ptext:dst variants of it in the 153 barrier or dependency description semantics, so two bits might need to be 154 added to describe both internal->external and external->internal 155 transitions. 156 Transitioning a resource to a special queue family corresponds well with the 157 operation of transitioning to a separate Vulkan instance, in that both 158 operations ideally include scheduling a barrier on both sides of the 159 transition: Both the releasing and the acquiring queue or process. 160 Using a special queue family requires adding an additional reserved queue 161 family index. 162 Re-using ename:VK_QUEUE_FAMILY_IGNORED would have left it unclear how to 163 transition a concurrent usage resource from one process to another, since 164 the semantics would have likely been equivalent to the currently-ignored 165 transition of 166 ename:VK_QUEUE_FAMILY_IGNORED{nbsp}->{nbsp}ename:VK_QUEUE_FAMILY_IGNORED. 167 Fortunately, creating a new reserved queue family index is not invasive. 168 169 Based on the above analysis, the approach of transitioning to a special 170 "`external`" queue family was chosen. 171 172 5) Do internal driver memory arrangements and/or other internal driver image 173 properties need to be exported and imported when sharing images across 174 processes or APIs. 175 176 *RESOLVED*: Some vendors claim this is necessary on their implementations, 177 but it was determined that the security risks of allowing opaque meta data 178 to be passed from applications to the driver were too high. 179 Therefore, implementations which require metadata will need to associate it 180 with the objects represented by the external handles, and rely on the 181 dedicated allocation mechanism to associate the exported and imported memory 182 objects with a single image or buffer. 183 184 6) Most prior interoperation and cross-process sharing APIs have been based 185 on image-level sharing. 186 Should Vulkan sharing be based on memory-object sharing or image sharing? 187 188 *RESOLVED*: These extensions have assumed memory-level sharing is the 189 correct granularity. 190 Vulkan is a lower-level API than most prior APIs, and as such attempts to 191 closely align with to the underlying primitives of the hardware and 192 system-level drivers it abstracts. 193 In general, the resource that holds the backing store for both images and 194 buffers of various types is memory. 195 Images and buffers are merely metadata containing brief descriptions of the 196 layout of bits within that memory. 197 198 Because memory object-based sharing is aligned with the overall Vulkan API 199 design, it exposes the full power of Vulkan on external objects. 200 External memory can be used as backing for sparse images, for example, 201 whereas such usage would be awkward at best with a sharing mechanism based 202 on higher-level primitives such as images. 203 Further, aligning the mechanism with the API in this way provides some hope 204 of trivial compatibility with future API enhancements. 205 If new objects backed by memory objects are added to the API, they too can 206 be used across processes with minimal additions to the base external memory 207 APIs. 208 209 Earlier APIs implemented interop at a higher level, and this necessitated 210 entirely separate sharing APIs for images and buffers. 211 To co-exist and interoperate with those APIs, the Vulkan external sharing 212 mechanism must accommodate their model. 213 However, if it can be agreed that memory-based sharing is the more desirable 214 and forward-looking design, legacy interoperation considerations can be 215 considered another reason to favor memory-based sharing: While native and 216 legacy driver primitives that may be used to implement sharing may not be as 217 low-level as the API here suggests, raw memory is still the least common 218 denominator among the types. 219 Image-based sharing can be cleanly derived from a set of base memory- object 220 sharing APIs with minimal effort, whereas image-based sharing does not 221 generalize well to buffer or raw-memory sharing. 222 Therefore, following the general Vulkan design principle of minimalism, it 223 is better to expose even interopability with image-based native and external 224 primitives via the memory sharing API, and place sufficient limits on their 225 usage to ensure they can be used only as backing for equivalent Vulkan 226 images. 227 This provides a consistent API for applications regardless of which platform 228 or external API they are targeting, which makes development of multi-API and 229 multi-platform applications simpler. 230 231 7) Should Vulkan define a common external handle type and provide Vulkan 232 functions to facilitate cross-process sharing of such handles rather than 233 relying on native handles to define the external objects? 234 235 *RESOLVED*: No. 236 Cross-process sharing of resources is best left to native platforms. 237 There are myriad security and extensibility issues with such a mechanism, 238 and attempting to re-solve all those issues within Vulkan does not align 239 with Vulkan's purpose as a graphics API. 240 If desired, such a mechanism could be built as a layer or helper library on 241 top of the opaque native handle defined in this family of extensions. 242 243 8) Must implementations provide additional guarantees about state implicitly 244 included in memory objects for those memory objects that may be exported? 245 246 *RESOLVED*: Implementations must ensure that sharing memory objects does not 247 transfer any information between the exporting and importing instances and 248 APIs other than that required to share the data contained in the memory 249 objects explicitly shared. 250 As specific examples, data from previously freed memory objects that used 251 the same underlying physical memory, and data from memory obects using 252 adjacent physical memory must not be visible to applications importing an 253 exported memory object. 254 255 9) Must implementations validate external handles the application provides 256 as input to memory import operations? 257 258 *RESOLVED*: Implementations must return an error to the application if the 259 provided memory handle cannot be used to complete the requested import 260 operation. 261 However, implementations need not validate handles are of the exact type 262 specified by the application.