We have all used a camera. In fact, with the rise of TikTok and Instagram, images are the rage of the day. Billions of photos and videos are made every day and uploaded on the internet. But have you ever wondered how your smartphone’s camera works?
A digital camera is made of some lens and a sensor (and many other technical things that we do not care about) – See Figure 1. The sensor is in fact a grid of light-sensitive detectors that convert light into digital image data. Each detector usually corresponds to one “pixel” of the image. To make a video, the sensor must take multiple images called “frames” (usually 24 to 60 per second), and then stitch them together into one continuous “movie”. Advancements in technology have made this complicated device cheap to make so that we have decent cameras even in affordable smartphones. But we cannot use such cameras to capture processes in say, living cells in a microscope, which occur in the billionth of a second! Or those that emit light over a wider spectrum than the red, green and blue (RGB) typical of a colour photo. For imaging such phenomena, we need special detectors which are fast and can even capture invisible light such as infrared. Unfortunately, they do not come cheap; making a grid of special detectors costs much more than a commercialgrade camera sensor.
Single-pixel imagining
This is where this new technique of single-pixel imaging comes in. In this method, instead of using an expensive grid of detectors, light from the entire scene is collected with a lens into a single, high-performance detector. The scene is illuminated with special, checkered patterns (See Fig. 2) instead of plain light so that details of the object are preserved. Taking multiple images with different patterns and doing a mathematical computation allows us to retrieve the original image, now enriched with the added time and spectrum data. The simple reason this works is the principle of compression. A typical photo that we capture with our smartphone camera is stored as a JPEG, which is a highly compressed format for storing the actual digital data captured by the sensor. This is possible because the latter has a lot of redundant information.
A single-pixel camera (SPC) tries to do the reverse: it directly captures the compressed information from the scene, by illuminating only parts of it at a time. This is known as Compressed Sensing.
Thus, an advantage of this method, apart from the lower cost, is that we can decide the level of compression in advance. A smaller number of patterns means more compression and faster acquisitions; but with lower image quality. More patterns lead to better quality but takes longer to capture. One main goal of the CONcISE project is to improve the speed and reliability of this method, to make it usable for imaging live tissues during surgeries. The main challenges are choosing the most optimal patterns for illuminating the scene and computing the real image in a reasonable time. We have recently discovered that using Machine Learning (the same technique that powers modern AI apps) can improve the performance of the SPC on both fronts. Currently, we are experimenting with integrating this new approach with the traditional single-pixel imaging microscope.
