What are good algorithms for vehicle license plate detection?

C#Image Computer Vision Ocr Object Detection

C# Problem Overview

Background

For my final project at university, I'm developing a vehicle license plate detection application. I consider myself an intermediate programmer, however my mathematics knowledge lacks anything above secondary school, which makes producing the right formulas harder than it probably should be.

I've spend a good amount of time looking up academic papers such as:

When it comes to the math, I'm lost. Due to this testing various graphic images proved productive, for example:

alt text

However this approach only worked to that particular image, and if the techniques were applied to different images, I'm sure a poorer conversion would occur. I've read about a formula called the "bottom hat morphology transform", which does the following:

> Basically, the trans- formation keeps all the dark details of the picture, and eliminates everything else (including bigger dark regions and light regions).

I can't find much information on this, however the image within the documentation near the end of the report shows its effectiveness.

Other constraints

Developing in C#
Confining the project to UK registration plates only
I can choose the images to convert as a demonstration

Question

I need advice on what transformation techniques I should focus on developing, and what algorithms can help me.

EDIT: New information present on https://stackoverflow.com/questions/4727119/continued-vehicle-license-plate-detection

C# Solutions

Solution 1 - C#

There are a number of approaches you can take but the first strategy that pops into mind is to:

Discovery/research: Identify the set of colors and fonts that you may need to identify. If your sample picture is representative of most British plates then your job is made easier. E.g. Simple, singular font and black lettering on a white background
Code: Attempt to identify a rectangular region of an image where the colors are predominantly white and black. This is not a terribly math-heavy problem and it should give you the license plate region to concentrate on.
Code: Do some clean up on your subregion such conversion to pure black and white (monochrome) and perhaps scaling/shifting into a nice, tight rectangle.
Use API: Next employ an existing OCR (optical character recognition) algorithm on your sub-selected image region so see if you can read the text.

Like I said, this is one strategy of many but it comes to mind as one requiring the least amount of heavy math... that is if you can find an OCR implementation that will work for you.

Solution 2 - C#

I did a similar project a few years ago in Java, first I applied the Sobel operator and then masked all the image with an image of a plate (with the Sobel operator applied too). The region of maximum coincidence is where the plate is. Then apply an OCR to the selected region to get the number.

Solution 3 - C#

Here's how I suggest you should do this task. Read my detailed answer here.

Convert to Grayscale.
Gaussian Blur with 3x3 or 5x5 filter.
Apply Sobel Filter to find vertical edges.

Sobel(gray, dst, -1, 1, 0)
Threshold the resultant image to get a binary image.
Apply a morphological close operation using suitable structuring element.
Find contours of the resulting image.
Find minAreaRect of each contour. Select rectangles based on aspect ratio and minimum and maximum area.
For each selected contour, find edge density. Set a threshold for edge density and choose the rectangles breaching that threshold as possible plate regions.
Few rectangles will remain after this. You can filter them based on orientation or any criteria you deem suitable.
Clip these detected rectangular portions from an image after adaptiveThreshold of the original (grayscale) image and apply OCR.

Solution 4 - C#

It tells you exactly how to compute the bottom hat transform(sort of looks like an inverted graduated threshold transform to me).

First thing to do is implement the two morphology functions dilation and erosion.

To do this you need your f and b then you compute the function over a small region of the image at a point keeping the largest value found.

(f ⊕ b)(s, t) = max{f (s − x, t − y) + b(x, y)
|(s − x), (t − y) ∈ Df ; (x, y)∈Db}

What this says is, take the maximum of the expression over all points in the domain region(such as a small rectangle centered at your point (s,t).

simple pseudo code would be

max = -infinity // for the point (s,t) on the image, must compute this for all points
for(x = -5 to 5)
for(y = -5 to 5)
max = Max(max, f(s - x, t - y) + b(x,y))

effectively we now have a new image of the max values.

It's actually quite simple so don't make it harder than it is(we are simply adding b(x,y) to each point in the region and finding out which one gives the maximum value).

you do the same for the erosion(very similar to above)

Now the opening and closing is the composition of the two

You can think of it first as performing a dilation and then an erosion for an opening.

It says finally subtract the closing from the original image and you should have your transform.

Solution 5 - C#

UK already has a system that does that. I recall seeing a TV show in which they demonstrated they can find a car within London within 10 minutes (assuming they know the number and the car is driving around) Just reading Wikipedia gives you the pointers you need to start thinking about the issue: http://en.wikipedia.org/wiki/Automatic_number_plate_recognition

Solution 6 - C#

If you're interested in the problem of detecting the presence of a license plate (as opposed to recognizing it), you should probably look at the text detection in images as it is related to what you're doing.

This question is related to yours: https://stackoverflow.com/questions/4606274/algorithm-to-detect-presence-of-text-on-image/4609739#4609739

Solution 7 - C#

You can also refer to Automatic License Plate Recognition library & this query. This will also give you some idea about how to approach things,and how existing solutions are.

But as answered by paul, you should first try to find the rectangular number plate from the complete image and then binarize it and then use the OCR libraries available (Tesseract would be recommended)

You can refer to this link which will help you find the rectangular plate. You need to use openCV libraries, so you will not need alot of math but yes a basic understanding of what is happening behind the scenes can help you solve the problem in better way.

Solution 8 - C#

I would suggest using a service or third party for this. Open ALPR, provides an open source package that is very accurate for this service.

Open ALPR - https://www.openalpr.com/

Demo of Video Open ALPR

https://www.youtube.com/watch?v=E-U_H9EbW60

Or you could use an API -

Macgyver Computer Vision API

https://askmacgyver.com/explore/program/license-plate-recognition/3X5D3d2k

In this API you would simply make a post request to -

https://macgyver.services

Example Payload

{
id: "3X5D3d2k",
key: "free",
data: {
    "image_url": "https://storage.googleapis.com/marketing-files/program-markdown-assets/license-detection/license-plate.jpg",
    "country": "us",
    "numberCandidates": 2
  }
}

The image above would return the following -

"plate": "284FH8"
"confidence": 90.601013

Content Type	Original Author	Original Content on Stackoverflow
Question	Ash	View Question on Stackoverflow
Solution 1 - C#	Paul Sasik	View Answer on Stackoverflow
Solution 2 - C#	rodrigoap	View Answer on Stackoverflow
Solution 3 - C#	Abdul Fatir	View Answer on Stackoverflow
Solution 4 - C#	AbstractDissonance	View Answer on Stackoverflow
Solution 5 - C#	Mircea	View Answer on Stackoverflow
Solution 6 - C#	carlosdc	View Answer on Stackoverflow
Solution 7 - C#	Mayank	View Answer on Stackoverflow
Solution 8 - C#	Timothy Moody	View Answer on Stackoverflow