UE5.1对移动端延迟渲染做了一波大更新，在只使用3张gbuffer(不算SceneColor和Depth)的情况下支持了桌面端全部的shadingmodel，并且对Vulkan、Metal甚至是GLES都做了On Chip Memory的带宽优化。正好前面我刚折腾完Unity 2021.3的延迟管线，就顺便看看UE5.1这边有什么新花样。

官方talk:

虚幻5的移动端延迟渲染技术到底有多好用？ - 腾讯游戏学堂的文章 - 知乎
https://zhuanlan.zhihu.com/p/562673914

经测试&查阅源码后确认支持的feature

  1. directional light可以有多盏(必须位于不同light channel), 只有一盏可以投射阴影
  2. point light数量不再有限制, 不支持阴影
  3. spot light可以投射阴影
  4. 每个light均只能有一个light channel
  5. 物体可以有多个light channel(即可以同时受多个light channel光源影响)
  6. 半透明效果正常
  7. gles/vulkan均支持On Chip Memory优化, 且在gles上根据硬件不同选择framebuffer fetch/pixel local storage方案(Mali平台的PLS未测试, Metal平台未测试)
  8. 在关闭静态光的情况下, 使用三张gbuffer支持了桌面端全部的shadingmodel, 并在源码里预留了第四张gbuffer的开关(未启用)

测试场景

  • 六个点光源: 左侧球体周围四个, channel2, 右侧椅子上各一个, channel0
  • 一个聚光: 左侧球体左边, channel1, 投射阴影
  • 两个方向光: 主光白色, channel0, 副光红色, channel1
  • 左侧球体channel012, 左侧地板channel01, 右侧地板及其他物体均为channel0

延迟渲染相关源码分析

API兼容

因为兼容了各种API和平台的On-Chip Memory优化方案, 所以在Basepass里通过宏来做处理.

USE_GLES_FBF_DEFERRED和MOBILE_EXTENDED_GBUFFER在ShaderCompiler.cpp中被定义:

这里的USE_GLES_FBF_DEFERRED仅决定了是否为GLES平台, 而FBF/PLS的判断则在另外的地方, 之后再详细说明.

LightPass中gbuffer的读取是On-Chip Memory优化的核心, 这部分代码被定义在MobileDeferredShading.usf中:

可以明确看到区分了Vulkan/Metal/GLES的情况, 以及在最坏情况下默认实现就是传统的MRT.

和subpass相关gbuffer读取被定义在了这三个文件中:

打开vulkan的看了下, 就是用HLSL写shader然后用DXC编译给vk用的正常写法.

至于决定是否将深度渲染到RT的USE_SCENE_DEPTH_AUX, 其shader端的定义如下, 对于移动延迟来说, Metal/GLES开启.

C++端在FMobileSceneRenderer::RenderDeferred()函数中通过bRequiresSceneDepthAux判断:

bRequiresSceneDepthAux的值依赖于函数MobileRequiresSceneDepthAux():

GLES的FrameBuffer Fetch/Pixel Local Storage

至于gles自己的PLS和FBF的区分, 通过用renderdoc对gles真机(Anreno平台)包抓帧得到的glsl代码可以看到以下代码结构(省略大量无关计算代码):

Basepass:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
#version 320 es
#define UE_MRT_FRAMEBUFFER_FETCH 1
#ifdef UE_MRT_FRAMEBUFFER_FETCH
 
    //...
    layout(location = 0) out vec4 out_var_SV_Target0;
    layout(location = 1) out vec4 out_var_SV_Target1;
    layout(location = 2) out vec4 out_var_SV_Target2;
    layout(location = 3) out vec4 out_var_SV_Target3;
 
    #ifndef GL_ARM_shader_framebuffer_fetch_depth_stencil
        layout(location = 4) out highp float out_var_SV_Target4;
    #endif
    //...
 
    int main()
    {
        //...
        out_var_SV_Target0 = vec4(_335.x, _335.y, _335.z, _157.w);
        out_var_SV_Target1 = _103;
        out_var_SV_Target2 = _104;
        out_var_SV_Target3 = _105;
        #ifndef GL_ARM_shader_framebuffer_fetch_depth_stencil
            out_var_SV_Target4 =gl_FragCoord.z;
        #endif
    }
 
#else // UE_MRT_FRAMEBUFFER_FETCH
    __pixel_local_outEXT _PLSOut    // PLS, 因为是第一个pass, 不输入只输出, 所以用out修饰
    {
        layout(rgb10_a2) mediump vec4 out_var_SV_Target0;
        layout(rgba8) mediump vec4 out_var_SV_Target1;
        layout(rgba8) mediump vec4 out_var_SV_Target2;
        layout(rgba8) mediump vec4 out_var_SV_Target3;
    };
    #ifndef GL_ARM_shader_framebuffer_fetch_depth_stencil   // 是否支持depth的fbf
        layout(location = 4) out highp float out_var_SV_Target4;
    #endif
    //...
    int main()
    {
    //...
        out_var_SV_Target0 = vec4(_335.x, _335.y, _335.z, _157.w);
        out_var_SV_Target1 = _103;
        out_var_SV_Target2 = _104;
        out_var_SV_Target3 = _105;
        #ifndef GL_ARM_shader_framebuffer_fetch_depth_stencil
            out_var_SV_Target4 =gl_FragCoord.z;
        #endif
    }
#endif

LightPass:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
#version 320 es
#define UE_MRT_FRAMEBUFFER_FETCH 1
#ifdef UE_MRT_FRAMEBUFFER_FETCH
 
    //... 
    highp vec4 GENERATED_SubpassFetchAttachment1;
    highp vec4 GENERATED_SubpassFetchAttachment2;
    highp vec4 GENERATED_SubpassFetchAttachment3;
    //... 
    layout(location = 0) out vec4 out_var_SV_Target0;
    layout(location = 1) inout vec4 out_var_SV_Target1;     //注意被修饰为inout, 既为fbf的语法
    layout(location = 2) inout vec4 out_var_SV_Target2;
    layout(location = 3) inout vec4 out_var_SV_Target3;
    //... 
    #if !defined(GL_ARM_shader_framebuffer_fetch_depth_stencil) && defined(GL_EXT_shader_framebuffer_fetch)
        layout(location = 4) inout highp vec4 out_var_SV_Target4;
    #endif
    float GLFetchDepthBuffer()
    {
        #if defined(GL_ARM_shader_framebuffer_fetch_depth_stencil)
            return gl_LastFragDepthARM;
        #elif defined(GL_EXT_shader_framebuffer_fetch)
            return out_var_SV_Target4.x;
        #else
        return 0.0f;
        #endif
    }
 
    void main()
    {
        GENERATED_SubpassFetchAttachment1 = out_var_SV_Target1;
        GENERATED_SubpassFetchAttachment2 = out_var_SV_Target2;
        GENERATED_SubpassFetchAttachment3 = out_var_SV_Target3;
        highp vec4 _609 = GENERATED_SubpassFetchAttachment1;
        highp vec4 _611 = GENERATED_SubpassFetchAttachment2;
        highp vec4 _613 = GENERATED_SubpassFetchAttachment3;
        highp float _621 = (int(1u != 0u) == 0) ? 0.0 : GLFetchDepthBuffer();
        //...
        out_var_SV_Target0 = vec4(_1505.x, _1505.y, _1505.z, _573.w);
    }
 
#else // UE_MRT_FRAMEBUFFER_FETCH
    __pixel_localEXT _PLSOut    // PLS, 既输入又输出
    {
        layout(rgb10_a2) highp vec4 GENERATED_SubpassFetchAttachment0;
        layout(rgba8) highp vec4 GENERATED_SubpassFetchAttachment1;
        layout(rgba8) highp vec4 GENERATED_SubpassFetchAttachment2;
        layout(rgba8) highp vec4 GENERATED_SubpassFetchAttachment3;
    };  
    #if !defined(GL_ARM_shader_framebuffer_fetch_depth_stencil) && defined(GL_EXT_shader_framebuffer_fetch)
        layout(location = 4) inout highp vec4 out_var_SV_Target4;
    #endif
 
    float GLFetchDepthBuffer()
    {
        #if defined(GL_ARM_shader_framebuffer_fetch_depth_stencil)
            return gl_LastFragDepthARM;
        #elif defined(GL_EXT_shader_framebuffer_fetch)
            return out_var_SV_Target4.x;
        #else
            return 0.0f;
        #endif
    }
 
    void main()
    {
        highp vec4 _609 = GENERATED_SubpassFetchAttachment1;
        highp vec4 _611 = GENERATED_SubpassFetchAttachment2;
        highp vec4 _613 = GENERATED_SubpassFetchAttachment3;
        highp float _621 = (int(1u != 0u) == 0) ? 0.0 : GLFetchDepthBuffer();
        //...
        GENERATED_SubpassFetchAttachment0 += vec4(_1505.x, _1505.y, _1505.z, _573.w);
    }
#endif

可以很直观的看到GLSL的代码里根据UE_MRT_FRAMEBUFFER_FETCH区分了fbf和pls, 查阅源码可以发现该宏是在定义在OpenGLShaders.cpp中的:

除此文件以外, OpenGLShaderCompiler.cpp GlslBackend.cpp也均与GLSL代码的生成相关, 可以自行查阅.

此外, 在用RenderDoc分析gles包时, 发现LightPass的input栏中看不到BasePass算好的gbuffer, 猜测应该就是fbf的特性所致.

Vulkan真机RenderDoc分析

Vulkan真机与Unity近似, 同样也都用上了renderpass/subpass的优化:

图中为空的subpass1是贴花Decal:

也可以通过Resource Inspector看到Renderpass的各种具体参数, 比如这里, attachemnt[1]既GBufferA的loadOp和storeOp分别为Clear和Dont Care.

ShadingModel相关

移动端用于encode/decode gbuffer的函数定义在了DeferredShadingCommon.ush中, 针对不同的shadingmodel均做了区分, 必须要通过MOBILE_SHADINGMODEL_SUPPORT来开启多shadingmodel的支持(不然就只默认Default Lit):

而移动端延迟渲染下的MOBILE_SHADINGMODEL_SUPPORT则依赖于ENABLE_SHADINGMODEL_SUPPORT_MOBILE_DEFERRED:

ENABLE_SHADINGMODEL_SUPPORT_MOBILE_DEFERRED在MobileBasePassRendering.h中被定义:

其依赖的函数MobileUsesGBufferCustomData()与MobileUsesExtenedGBuffer()一样被定义在RenderUtils.cpp中:

可以确认UE5.1移动端延迟渲染shadingmodel的实现, 确实是在关闭静态光的情况下, 通过将Custom Data写入写入GBuffer中与静态光相关的通道来实现的.

此外, shader里定义了MOBILE_EXTENDED_GBUFFER用于定义额外的第四张GBufferD, 其定义依赖于定义在RenderUtils.cpp里的函数MobileUsesExtenedGBuffer():

基本就和官方那篇talk说的一致, gles是肯定没法开GBufferD的, vk得看具体硬件支持, 而metal那边因为硬件统一只需要A8以上就OK.

不过目前直接在代码里写死false了, 应该是留给以后或用户自己来扩展? 等将来把源码拉下来改一改试试.

TL; DR

UE5.1的移动端延迟渲染管线在关闭静态光的情况下支持了桌面端全部的shadingmodel, 并且不止是vulkan和metal, 就连gles全都用上了On-Chip Memory的带宽优化(FBF/PLS), 相比只支持vulkan/metal的Unity强了不少.

而且经测试UE5.1的延迟渲染管线相比Unity2021.3 URP的延迟管线更加稳定好用, 坑少了很多, 兼容性也更好.

具体的扩展性与实用性还要等UE5.1 release后拉取源码版修改后测试.