¶ Captured image post-processing
Sometimes, captured image is not good enough to do image/color/ml search straight away - it could be too noisy or contain too much similar colors and objects to make analysis effects.
In this case, post-processing via one of available effects is a way to go. You can:
- remove or replace colors similar to specified hue
- grayscale/negate the image, to make it easier to recognize text
- sharpen edges to make it easier for ML to detect paths
- erode and dilate the image to amplify some color
- resize to letterbox to make ML-processing easier
- and much more!
¶ Performance
All effects are highly optimized and are expected to be applied in real-time to captured images. Some of them, such as color removal, operate at 500+ fps, meaning that you can apply multiple of them without noticing major difference in processing times.
If you'll notice that effect greatly affects performance - please report, as this is probable a bug.
¶ Available effects
Let's take a look at what effects can do. Here is an initial image:
¶ Color Removal
Can remove(replace with black) colors similar to specified one using HSV-scale.
For example, you can easily remove ground and leave only characters and monsters - this will make training target-picking ML-model a breeze
¶ Color Replacement
Works almost exactly like Color Removal, with the one exception that it allows you to specify any color which will be used as replacement.
Also it can replace everything which is NOT similar to specified color.
For example, by applying terrain replacement effect where passable terrain is in white and non-passable is in black, you can build a movement system around it.
P.S. Note that any effect could be applied to part of the captured image
¶ Crop
It is a simple effect which allows to crop the image to desired region.
¶ Denoise
Before:
After:
¶ Dilate
Dilations increase the size of foreground objects and are especially useful for joining broken parts of an image together.
¶ Erode
Just like water rushing along a river bank erodes the soil, an erosion in an image “erodes” the foreground object and makes it smaller. Simply put, pixels near the boundary of an object in an image will be discarded, “eroding” it away.
¶ Gaussian Blur
Blurs part of the image, could be useful to remove smaller elements such as text, to make further post-processing and/or analysis more effective
¶ Grayscale
Turns people black and white. Plain and simple. In some cases it is more effective to get rid of coloring to avoid having to remove undesired elements
¶ Negate
Negates ALL colors of an image. Black turns white and vice versa. The most useful application of this is to turn white text or text of any other color into black as Text recognition works the best when there are Black letters on White canvas.
¶ Rescale, %
Could be used to reduce size of a captured image to reduce computation power required to process the image. For example, image of size 1920x1080 contains 2,073,600 pixels. Resize it to 50% and you'll get 960x540 which is only 518,400 pixels! This is less important for ML, but for classical algorithms which are used in Image Search and Color Search this could give them a great boost
¶ Resize, in pixels
Allows to resize image to specified size. This is especially useful when you're working with machine-learning models, which are trained on a specific set of images. By applying letterboxing, you can keep dimensions intact and make models work better.
¶ Binary Threshold
This effect converts image to black-and-white and highlights some othe those.
Play around with different separation techniques to pick which one will be better for your specific case
The most detailed guide on this has OpenCV. It is for Python but has a lot of illustrations.
https://learnopencv.com/opencv-threshold-python-cpp/
https://medium.com/@manik.butex41/an-introduction-to-thresholding-operations-in-opencv-64aff0cb2826
¶ Color Quantization (alpha)
This effect reduces number of colors present in the image. Makes further analysis more simple.