Create SKSurface from GRBackendRenderTarget to show Vulkan 3D graphics

[abenedik]

What would you like to be added

We were able to provide a basic support for Uno platform for Ab4d.SharpEngine - Vulkan based 3D rendering engine. This means that 3D scene that is rendered by Ab4d.SharpEngine can be shown in an Uno platform app (using Skia backend).

Currently, this is done by copying the bytes from a rendered texture to a SKBitmap and then showing this in a class that is derived from SKCanvasElement.

This works but is very inefficient because the rendered texture is copied from the GPU memory to the CPU memory (to SKBitmap) and then Skia sends this back to the GPU to show it in the app.

It would be much better to preserve the rendered texture on the GPU and somehow tell Skia to show that (this is also possible by Avalonia - see AvaloniaUI/Avalonia#9925).

I tried to achieve that by creating a GRBackendRenderTarget object from a Vulkan texture. But when I call SKSurface.Create method and pass the GRBackendRenderTarget, no surface is created.

See the source code for that:
https://github.com/ab4d/Ab4d.SharpEngine.Samples/blob/features/UnoPlatformSharedTexture/Ab4d.SharpEngine.Samples.UnoPlatform/Ab4d.SharpEngine.Samples.UnoPlatform/SharedTextureTestPage.xaml.cs

This can be tested by cloning the "features/UnoPlatformSharedTexture" branch of the Ab4d.SharpEngine.Samples on GitHub (https://github.com/ab4d/Ab4d.SharpEngine.Samples/tree/features/UnoPlatformSharedTexture). After cloning, open the Ab4d.SharpEngine.Samples.UnoPlatform.sln solution file in the Ab4d.SharpEngine.Samples.UnoPlatform folder.

In the SharedTextureTestPage.xaml.cs file there are comments where I was investigating what is going on when the SKSurface.Create method is called. It seems that this calls sk_surface_new_backend_render_target (from SkiaSharp) and then native WrapBackendRenderTarget is called. This then does some validations if the GrBackendRenderTarget is compatible with the GrRecordingContext. I assume that one of this checks returns false and this returns null from the Create method.

Maybe the problem is that I have created an ad-hoc context by calling GRContext.CreateGl(). It would be probably better to use the context of the App, but I do not know if it is possible to get that.

I do not know how to proceed. It would be great if someone with a better knowledge of the Uno platform internals could check this and provide a way to show a shared texture.

This would add Uno platform as another platform that is fully supported by Ab4d.SharpEngine. This would also allow any other third-party rendering engine to show its rendered textures in Uno platfom.

Note:
To test how the integration with SKBitmap works (coping to CPU), start the app by showing the MainPage - uncomment the line 44 and comment line 45 in App.xaml.cs.

Why is this needed

No response

For which platform

Skia (WPF), Skia (Linux X11)

Anything else we need to know?

Note:
Ab4d.SharpEngine is a commercial library. However the next version will allow getting a free license for open-source projects.

[jeromelaban]

Thanks for the report!

In practice, you're using the right integration point with SKCanvasElement, but the reason why the interop is not working is not clear at this point.

Would you be able to provide a sample of this specific scenario using SkiaSharp directly, maybe using a blank WPF app? There may be some creation sequence in Uno that makes it incorrect that we could fix to get this scenario running.