How to Get Started with Metal APIs Using UIView and SwiftUI

Background

Metal is a powerful and low-level framework for graphics and computer programming on Apple platforms. It allows you to take full advantage of the performance of the GPU (Graphics Processing Unit) in order to perform complex computations and rendering tasks.

One of the main use cases for Metal is image processing, as it allows you to perform operations on images and videos in real time with high performance and low overhead.

In this blog post, we will dive into the world of Metal by drawing a triangle.

We will cover the basics of Metal programming, including the main concepts and terminology, as well as the steps for setting up your development environment.

By the end of this blog, you should have a solid understanding of how to use Metal to create simple 3D graphics, and be ready to take on more advanced Metal projects. We will end up creating an app that draws this triangle.

Sponsored

We are what we repeatedly do. Excellence, then, is not an act, but a habit. Try out Justly and start building your habits today!

Setting up Metal in an iOS Project

To use Metal APIs in an iOS project, you will need to perform the following steps:

1. Create a new Xcode project and select “iOS” as the platform.
2. In the project settings, go to the “General” tab and make sure that your project is targeting a version of iOS that supports Metal (Metal is supported on iOS 8 and later).
3. In the “Build Settings” tab, search for “Metal” and make sure that the “MetalCaptureEnabled” option is set to “Yes”. This will enable the Metal APIs for your project.
4. In your code, import the Metal framework by adding the following line at the top of your file: import Metal
5. To use Metal in your code, you will need to create a MTLDevice object, which represents the device (usually the GPU) on which Metal will run.

You can create a MTLDevice object like this:

let device = MTLCreateSystemDefaultDevice()

Once you have a MTLDevice object, you can create other Metal objects, such as MTLCommandQueue and MTLCommandBuffer, to perform operations on the device.

Now we are ready to explore Metal concepts.

Understanding Metal Shaders Usage

In Metal, a shader is a small program that runs on the GPU and is responsible for performing a specific task, such as rendering a 3D model or applying an image filter.

There are three main types of shaders in Metal: vertex shaders, fragment shaders, and compute shaders.

1. Vertex shaders

Vertex shaders are responsible for transforming the 3D coordinates of a model into 2D screen space coordinates. They take a set of vertex data (such as position, normal, and texture coordinates) as input and give a set of transformed vertex data as output.

2. Fragment shaders

Fragment shaders are responsible for determining the final color of each pixel on the screen. They take the interpolated vertex data from the vertex shader as input and return a final color for each pixel as output.

3. Compute shaders

Compute shaders are a more general-purpose type of shader and can perform any type of computation on the GPU. They are often used for tasks such as image processing and simulations.

Brief Intro about Creating and Using Shaders in Metal

In Metal, shaders are written in a language called Metal Shading Language (MSL).

You can create a new MSL file in Xcode by selecting “File” > “New” > “File” and choosing “Metal Shading Language File” under the “Metal” section.

Once you have written your shaders, you will need to load them into your Metal project, you can do this using the MTLLibrary.

Let’s see a code snippet that demonstrates it.

// Create a Metal device
let device = MTLCreateSystemDefaultDevice()

//Create Metal library from MSL file
let metalLibrary = device.makeDefaultLibrary()

//load the vertex function from the library
let vertexFunction = metalLibrary.makeFunction(name: "vertexShader")

//load the fragment function from the library
let fragmentFunction = metalLibrary.makeFunction(name: "fragmentShader")

Understanding Pipelines and State

Now let’s see how we can attach the above functions to the pipeline state.

//create a render pipeline descriptor
let pipelineDescriptor = MTLRenderPipelineDescriptor()

//set the vertex and fragment functions
pipelineDescriptor.vertexFunction = vertexFunction
pipelineDescriptor.fragmentFunction = fragmentFunction

//create the pipeline state
let pipelineState = try device.makeRenderPipelineState(descriptor: pipelineDescriptor)

Now what is MTLRenderPipelineDescriptor ?

The MTLRenderPipelineDescriptor object contains properties such as the vertex function, fragment function, and depth/stencil state, which are used to configure the render pipeline state object.

makeRenderPipelineState(descriptor:) is a method of the MTLDevice class that creates a new MTLRenderPipelineState object from a MTLRenderPipelineDescriptor object.

This method takes a MTLRenderPipelineDescriptor object as its input and returns a MTLRenderPipelineState object that can be used to render 3D models or images.

Use Metal in a view controller

In this section, we will explore how to use Metal to display a Triangle in a UIView (iOS) or NSView (macOS) in your iOS or macOS application.

We will cover the main steps for setting up a Metal view, creating a Metal layer, and rendering a circle using Metal shaders.

1. Setting up the initial view

We’ll add a simple UIView to our main.storyboard and set height, width, and position according to our needs.

Now let’s take outlet of that view to our swift file and give it a name, here I’ve given a name mainView for a better understanding,

As we’ll add the created view to this view we need its outlet.

2. Setting up a Metal view

In order to use Metal, you will need to create a custom view that is backed by a Metal layer.

You can create a custom view by subclassing UIView and overriding the layerClass property to return CAMetalLayer class.

class MetalView: UIView {
    override class var layerClass: AnyClass {
        return CAMetalLayer.self
    }
}

Add its object to the main view controller class.

let metalView = MetalView()

3. Creating a Metal layer

Once you have set up your Metal view, you can access the underlying Metal layer by calling the layer property of the view.

The Metal layer is responsible for managing the GPU resources and presenting the rendered content.

let metalLayer = self.metalView.layer as? CAMetalLayer
metalLayer?.frame = .init(x: 0, y: 0, width: mainView.frame.width, height: mainView.frame.height)

Here we have set its frame with the parent view as we want to render the view on our added static view.

We also have to set the child view constraints with the parent view to locate the position.

mainView.addSubview(metalView)
metalView.topAnchor.constraint(equalTo: mainView.topAnchor).isActive = true
metalView.bottomAnchor.constraint(equalTo: mainView.bottomAnchor).isActive = true
metalView.leftAnchor.constraint(equalTo: mainView.leftAnchor).isActive = true
metalView.rightAnchor.constraint(equalTo: mainView.rightAnchor).isActive = true

We’ll add all these things in the viewDidLoad() method as we want to show the metal view with the view render.

4. Create a vertex buffer

To render a triangle using Metal, you will need to create a vertex buffer containing the vertex data for the triangle, and pass this data to the vertex shader.

let vertexData: [Float] = [0.0, 0.5, 0.0, 1.0,
                           -0.5, -0.5, 0.0, 1.0,
                           0.5, -0.5, 0.0, 1.0]
// create vertex buffer for circle
let vertexBuffer = device.makeBuffer(bytes: vertexData, length: vertexData.count * MemoryLayout<Float>.size, options: [])

I’m assuming you have added the device object after the declaration of the metalView object.

let device = MTLCreateSystemDefaultDevice()!

You can use the fragment shader to color the circle.

Now you may be wondering what are the Vertex shader and Fragment shader, so these are the metal file functions to create an image or view according to given in above definition.

Here is the implementation of both functions,

vertex float4 vertexShader(constant float3 *vertexArray [[ buffer(0) ]],
                           uint vid [[ vertex_id ]]) {
    float3 ver = vertexArray[vid];
    return float4(ver, 1.0);
}

fragment float4 fragmentShader() {
    return float4(1.0, 0.5, 0.5, 1.0);
}

We have to add these both functions in the .metal file, if you haven’t added it to your project then let’s please add it.

5. Creating a render pipeline state

In order to render the triangle, you will need to create a MTLRenderPipelineState object that contains the shaders and other state information used to render the triangle.

guard let drawable = metalLayer?.nextDrawable() else { return }

// create a render pipeline descriptor
let pipelineDescriptor = MTLRenderPipelineDescriptor()

// get defined metal functions from the metal file
let metalLibrary = device.makeDefaultLibrary()
let vertexFunction = metalLibrary?.makeFunction(name: "vertexShader")
let fragmentFunction = metalLibrary?.makeFunction(name: "fragmentShader")

// set the vertex function
pipelineDescriptor.vertexFunction = vertexFunction

// set the fragment function
pipelineDescriptor.fragmentFunction = fragmentFunction

// set the pixel format to the pipeline
pipelineDescriptor.colorAttachments[0].pixelFormat = drawable.texture.pixelFormat

do {
    // create the render pipeline state
    let pipelineState = try device.makeRenderPipelineState(descriptor: pipelineDescriptor)
} catch let error {
    print("Error: \(error.localizedDescription)")
}

This is not enough though.

6. Rendering the Triangle

Once you have set up your Metal view, Metal layer, pipeline state, and vertex buffer, you can render the triangle by creating a MTLCommandBuffer object, creating a MTLRenderCommandEncoder object, encoding the pipeline state, vertex buffer, and other state information, and then submitting the command buffer to the GPU.

Let’s add the given code in the do block.

// create a render command encoder
let renderPassDescriptor = MTLRenderPassDescriptor()
renderPassDescriptor.colorAttachments[0].texture = drawable.texture
renderPassDescriptor.colorAttachments[0].loadAction = .clear

// create a command buffer
let commandQueue = device.makeCommandQueue()!
let commandBuffer = commandQueue.makeCommandBuffer()!

// create a render command encoder
let renderEncoder = commandBuffer.makeRenderCommandEncoder(descriptor: renderPassDescriptor)!

// set the pipeline state
renderEncoder.setRenderPipelineState(pipelineState)

// set the vertex buffer
renderEncoder.setVertexBuffer(vertexBuffer, offset: 0, index: 0)

// render the triangle
renderEncoder.drawPrimitives(type: .triangle, vertexStart: 0, vertexCount: vertexData.count)

// end encoding
renderEncoder.endEncoding()

// present the drawable
commandBuffer.present(drawable)

// commit the command buffer
commandBuffer.commit()

Run the app.

Very easy right? The only thing you have to remember is the flow and order of things that we are adding.

Now let’s see how we can handle this in SwiftUI.

Use Metal in SwiftUI

In this section, we will explore how to use Metal in SwiftUI to display a triangle.

The only difference compared to UIView is how we get drawable that Metal uses for renderEncoder.present .

To use Metal in SwiftUI, we will need to use the MetalKit framework to create a MTKView. We can do this by wrapping the MTKView in a UIViewRepresentable struct and use it in your SwiftUI view.

struct TriangleView: UIViewRepresentable {
    func makeUIView(context: Context) -> MTKView {
        let view = MTKView()
        view.device = MTLCreateSystemDefaultDevice()
        view.delegate = context.coordinator
        return view
    }

    func updateUIView(_ uiView: MTKView, context: Context) {}

    func makeCoordinator() -> Coordinator {
        Coordinator()
    }

    class Coordinator: NSObject, MTKViewDelegate {

        func mtkView(_ view: MTKView, drawableSizeWillChange size: CGSize) {
        }

        func draw(in view: MTKView) {
          .
          .
          let descriptor = view.currentRenderPassDescriptor
          let renderEncoder = commandBuffer.makeRenderCommandEncoder(descriptor: descriptor)
          guard let drawable = view.currentDrawable else { return }
          .
          .
        }
}

In SwiftUI implementation apart from the above things everything will be the same as ViewController.

Here we are creating a drawable from the MTKView instead of the CAMetalLayer.

Let’s use this view in our ContentView to display a triangle,

struct ContentView: View {
    
    var body: some View {
        VStack {
            TriangleView()
                .frame(width: 230, height: 200, alignment: .center)
            Text("Hello, world!")
        }
        .padding()
    }
}

That’s it. We are done.

Conclusion

In this blog, we have covered the basics of using Metal APIs to display a triangle in a UIView and in SwiftUI. We have discussed how to set up a Metal view, create a Metal layer, and render a triangle using Metal shaders. We have also provided code snippets and tips to help you get started with using Metal in your own iOS or macOS application.

While this example is relatively simple, it demonstrates the power and flexibility of Metal APIs. With Metal, you can create complex and highly-optimized 3D graphics, image processing, and other GPU-accelerated tasks.

To continue learning about Metal, we recommend checking out the official Metal Programming Guide and the Metal Framework Reference from Apple.

Additionally, you can find many tutorials, sample codes, and other resources online that can help you learn more about Metal and how to use it in your own applications.

I hope this blog has been helpful in getting you started with using Metal APIs. Don’t hesitate to reach out if you have any further questions or need any help.

Also, we are planning to write the next part of this blog where we will demonstrate how we can create a few common image filters using Metal.

Stay tuned!!!

Similar Popular Articles