Line scan camera image processing for train photography

Summary

Line scan camera image processingI use my line scan camera to take cool pictures of trains and other stuff.But there’s a lot that goes into properly processing the images.FIGURE 2 A cool train, the Renfe AVE Class 102, nicknamed Pato because of its duck bill-like appearance.FIGURE 4 Nice CRH6A intercity electric multiple unit.1 About the cameraI’m using an Alkeria Necta N4K2-7C. It has a 4096×2 Bayer image sensor. I’m saving its raw data in 16 bit binary arrays.VIDEO2 Detecting the region of interestSometimes, I keep the line scan camera running for a while, and it generates tons of boring data of the background. To detect moving things, I compute an “energy function” that’s defined aswhere is the maximum pixel value of the image, and the partial derivative are the image gradient.This is because, for a static background, it will be full of horizontal stripes. By weighing the -direction (time direction) gradient against the total gradient norm, we can find areas where it’s a more vertical-ish structure rather than a horizontal structure. However, doing this by itself risks noisy gradients in empty (but noisy) areas where the gradient direction is completely random. The maximum pixel value term ensures that whatever gradient we see is salient.The image is divided into chunks and the score of a chunk is the 99th percentile energy.Finally, chunks containing moving objects are defined to be ones where the score is at least 1.5× that of the minimum score.This heuristic took me longer than I would like to admit to figure out. Previously, I came up with heuristics that worked well on one capture but couldn’t generalize well to other captures. Sometimes, the background will contain slowly moving foliage waving in the wind, that would screw up other methods of detection. That resulted in a lot of wasted time because time spent processing empty regions seriously slows down iteration speed when developing the later steps.3 Speed estimationThe most common question I get is, how ...