Pixel-Level Image Processing in Swift

If you search about image manipulation in Swift like I did, you might come across resources recommending CoreGraphics, MetalKit or GPUImage3. All of these options use the GPU to accelerate image processing, but they require a lot of prior knowledge.

Alternatively, you can explore CPU-based image processing, which is more accessible for beginners, albeit at the cost of performance. This route also provides an excellent opportunity to gain experience in working with 2D arrays.

Converting the Image to Pixels #

If you have a UIImage and want to manipulate each pixel, one way to do so is to convert the UIImage into a list of RGB values. We will convert the UIImage into a “[[[UInt8]]]”.

func getPixelData(image: UIImage) -> [[[UInt8]]]? {
    let size = image.size
    let dataSize = size.width * size.height * 4
    var pixelData = [UInt8](repeating: 0, count: Int(dataSize))
    let colorSpace = CGColorSpaceCreateDeviceRGB()
    let context = CGContext(data: &pixelData,
                            width: Int(size.width),
                            height: Int(size.height),
                            bitsPerComponent: 8,
                            bytesPerRow: 4 * Int(size.width),
                            space: colorSpace,
                            bitmapInfo: CGImageAlphaInfo.noneSkipLast.rawValue)
    guard let cgImage = image.cgImage else { return nil }
    context?.draw(cgImage, in: CGRect(x: 0, y: 0, width: size.width, height: size.height))
    
    var rgbArray = [[[UInt8]]]()
    
    let width = Int(size.width)
    let height = Int(size.height)
    
    for i in stride(from: 0, to: height, by: 1) {
        let current_row: [UInt8] = [UInt8](pixelData[(i*width*4)...i*(width)*4 + width * 4 - 1])
        rgbArray.append(current_row.chunked(into: 4))
    }
    
    return rgbArray
}

This chunked extension by hacking with swift is very convenient for this purpose.

extension Array {
    func chunked(into size: Int) -> [[Element]] {
        return stride(from: 0, to: count, by: size).map {
            Array(self[$0 ..< Swift.min($0 + size, count)])
        }
    }
}

Credit to Hacking with Swift The first two arrays contain the rows and columns, while the innermost array contains the RGBA values as 8 bit integers.

Applying a Transformation #

You can now modify the pixels as needed within the provided loop to edit the image, or create a new [[[UInt8]]] from the current image.

func transform(image:[[[UInt8]]]) -> [[[UInt8]]]? {
	let height = current.count
    let width = current[0].count
    for cols in 0...height-1{
	    for rows in 0...width-1{
		    // each pixel has a list of the rgba values
			let pixel: [UInt8] = image[cols][rows]
			// edit pixel here
		}
	}
	return image
}

Converting the pixels back to an Image #

func pixelToImage(pixelData: [[[UInt8]]]) -> UIImage?{
    let height = pixelData.count
    let width = pixelData[0].count
    let colorSpace = CGColorSpaceCreateDeviceRGB()
    
    var byteArr = Array(pixelData.joined().joined())
    let context2 = CGContext(data: &byteArr,
                            width: width,
                            height: height,
                            bitsPerComponent: 8,
                            bytesPerRow: 4 * width,
                            space: colorSpace,
                            bitmapInfo: CGImageAlphaInfo.noneSkipLast.rawValue)
    if context2 != nil{
        let img = context2?.makeImage()
        let finalImg = UIImage(cgImage: img!)
        return finalImg
    }
    return nil
}

While this approach is not the cleanest, I had to spend a lot of time finding a quick and easy way to change some pixels in an image. I wanted to share my findings in the hope that it can help others facing the same issue. Thanks for taking the time to read this, and I hope it proves useful to you.