What are the differences between CV_8U and CV_32F and what should I worry about when converting between them?

I have some code that is acting up and I suspect it's because I'm operating on the wrong types of data or converting between them poorly.

It is mixing cv::Mat objects of types CV_8U (which is what is created when reading a jpg as grayscale with cv::imread), CV_32F, and CV_32S.

What are the differences between these data types, and what do I need to be sure of when converting between them?

CV_8U is unsigned 8bit/pixel - ie a pixel can have values 0-255, this is the normal range for most image and video formats.

CV_32F is float - the pixel can have any value between 0-1.0, this is useful for some sets of calculations on data - but it has to be converted into 8bits to save or display by multiplying each pixel by 255.

CV_32S is a signed 32bit integer value for each pixel - again useful of you are doing integer maths on the pixels, but again needs converting into 8bits to save or display. This is trickier since you need to decide how to convert the much larger range of possible values (+/- 2billion!) into 0-255

Basically they just describe what the individual components are:

• CV_8U: 1-byte unsigned integer (unsigned char).

• CV_32S: 4-byte signed integer (int).

• CV_32F: 4-byte floating point (float).

What you always have to keep in mind is that you cannot just cast them from one into the other (or it probably won't do what you want), especially between differently sized types.

So always make sure you use real conversion functions for converting between them, like cv::convert or cv::Mat::convertTo. Don't just try to access the elements of e.g. a cv::Mat of CV_8U type using e.g. cv::Mat::at<float> or cv::Mat_<float>.

Or if you just want to convert individual elements and don't want to create a new matrix of the other type, access the elements using the appropriate function (in the example cv::Mat::at<unsigned char>) and convert the result to float.

Likewise is there also a difference between the number of components and a cv::Mat of CV_8UC3 type is different from an image of CV_8UC1 type and should (usually) not be accessed by cv::Mat::at<unsigned char>, but by cv::Mat::at<cv::Vec3b>.

EDIT: Seeing Martin's answer it may be that you are aware of this all and his explanations are more what you have been looking for.