Multi-frame Labeled Faces Database

Benchmark

This benchmark was created for the scientific purposes while this database was published and initial results were done by our team. These results are considered as a fundamentals for this research area and they are shown in leader-board table. There are 2,500 image sequences in the test set and all resulting images are compared with ground truth images and evaluated using several metrics. Mean of the values for given metric is used for representation as a single value result. Overall score is then computed as a sum of normalized metrics results. The lowest value is considered as better, in case observed metric where higher value is better (SSIM, PSNR), the normalized value is computed as 1 - norm(x). Following metrics are computed with min and max values for normalization:

Mean Square Error (MSE). Normalization min = 0, max = 500.
Structural Similarity (SSIM). Normalization min = 0, max = 1; (score = 1 -norm(x)).
Peak Signal to Noise Ratio (PSNR). Normalization min = 0, max = 50; (score = 1 -norm(x)).
Sharpness difference (CPBD). Normalization min = 0, max = 1.
Face feature extraction rate (FR failed) - using well-known FaceRecognition framework. This metric says how many predicted images failed to extract face features needed for face recognition (using CNN model). There is no mean for this metric. Normalization min = 0, max = 2500.

All metrics and overall score are computed automatically and detailed results are available via score hyperlink in leaderboard table. There is .json file containing means of all metrics and their normalized values. CSV file with all measurements will be send by email.

Attendance

The benchmark is publicly available and every team all around the world can attend it. Feel free to register and try to beat the best score by your model. The benchmark has few simple rules:

First, you have to register your team via form below, all fields are mandatory. Team name should be simple, without special characters and spaces. Number of members is not limited.
Select name of your method. You have to create a clear, simple method description (for example short text with references and graphical visualisation of the model) and upload it together with resulting images. For each submission you have to create new method description - at least highlight changes. The method description will be publicly available in leaderboard table. It must be in PDF format and name it "method.pdf".
For model training should be used only MLFDB train and validation set (with labels) but it is not limited only to this database. The aim of the benchmark is to be worldwide measurement for multi-frame super-resolution problem, therefore there is no limitation for training database usage.
When you are satisfied with your model performance, download private test set (without labels) and create predictions - resulting images in 64x64 px resolution. The name of images must correspond with test set order. It means for sequence 1 there will be result 1.JPG. Only .JPG extension is allowed. Finally place all resulting images into "images" folder.
Add your "method.pdf" and "images" folder into one .zip file (only .zip format is allowed) and submit it and wait for results. You will be notified by email when results are computed.
Number of submissions is limited to 10 according to avoid overfitting and customization for the private test set.

IMPORTANT: Submission form is currently under construction and it is not working. Please, send your resulting .zip file to martin.rajnoha@vutbr.cz for evaluation. Thanks for understanding.

Register
To attend MLFDB challenge you have to register your team.
.

Submit
- Prepare your method description and save as "method.pdf" file.
- All images must have ".JPG" extension and placed in "images" folder.
- Name of images corresponds with their order (1.JPG, 2.JPG, .. 2500.JPG).
- Add PDF file and "images" folder into .zip file with name of your team.
- Consider number of submissions is limited to 10.
- You will be notified by email immediately after results are computed.
.

Multi-frame Labeled Faces Database

Benchmark

Attendance

Register

Submit

Results

Leaderboard