Machine Learning Yearning 中文翻译ch1

1. 为什么需要机器学习策略？

机器学习是无数应用的基础，包括网页搜索，对抗垃圾邮件，语音识别，产品推荐及许多其它应用。假设你或者你的团队正在使用机器学习，并且希望快速取得进展。这本书将会帮助到你。

例子： 创立一家猫咪图片公司

假设你正在成立一家创业公司给爱猫人士提供无穷无尽的猫咪图片。你使用神经网络来建立一个计算机视觉系统用于检测图片种的猫。

但是悲剧的是你的学习算法准确度不够高。你面临着很大的压力来改善猫咪检测准确度。你应该怎么做？

你的团队提出了很多想法，比如：

• 获取更多的数据：获取更多的猫的图片。
• 或许更多元的训练集。比如在不同寻常位置的猫的图片，不寻常的颜色的猫图片，不同相机设定拍摄的猫图片。
• 通过增加梯度下降迭代次数来延长算法训练时间
• 尝试更大的神经网络，比如更多层，更多隐含单元，更多训练参数。
• 尝试更小的神经网络。
• 尝试加正则化（比如L2正则化）
• 改变神经网络的结构（激活函数，隐含单元数目等）
• 。。。

如果你能从这些方向中做出正确的选择，你的公司会成为一家领先的猫咪图片平台并走向成功。如果你选择错误，可能浪费了公司几个月的时间。你应该怎么做？

这本书会告诉你怎么做。大部分的机器学习问题都有一些线索告诉你哪些尝试是游泳的，哪些是没用的。学会阅读这些线索会节省你几个月甚至几年的开发时间。

Neural Networks for Low Level Image Processing

Until recently, machine learning (ML) or neural networks (NN) are mainly used in high level vision tasks, such as image segmentation, object recognition and detection. Low level image processing such as denoising, demosaicing, white balance still mainly rely on signal processing based methods which uses expert designed filters. There are usually a long list of filters in the whole processing pipeline which is run on a dedicated ISP chip. In the past one or two years, there are two new trends. One trend is that more and more researchers propose to apply NN for low level image processing and achieved fascinating performance in term of image quality and processing speed. The other trend is that neural network chip becomes more and more popular at various mobile platform, such as the latest Apple A11 Bionic chip and Huawei Kirin 970 chip. I believe in the near further, NN based methods will play important roles at some low level image processing tasks also some ISP chip may include some NN computing units.

This is a personal collection of works using neural networks for low level image processing. The list will be regularly updated. You are welcome to contribute. Papers of significance are marked in bold. My comments are marked in italic.

For a up-to-date list, please follow this github repository.

Table of Contents

• Review and comments
• Color constancy
• Denoising
• Demosaicing
• Superresolution
• Automatic adjustment
• Artefacts removal
• Pipeline
• Image quality evaluation
• Others

Review and comments

• Deep, Deep Trouble
  • This is an interesting comments from Michael Elad about the impact of Deep Learning on image processing, mathematics and humanity.
• Deep learning for image/video processing
  • Dr. Huang Yu’s summary of DL on image/video processing.

Color constancy

• Deep Specialized Network for Illuminant Estimation (ECCV, 2016, CUHK)
• Single and Multiple Illuminant Estimation Using Convolutional Neural Networks (TIP, 2017, Italy)
  • A three-stage method for illuminant estimation from RAW images
• Recurrent Color Constancy (ICCV, 2017, TUT)
  • Recurrent network for temporal color constancy using multiple frames instead of single frame
• FC4: Fully Convolutional Color Constancy with Confidence-weighted Pooling (CVPR, 2017, THU)
  • CNN based method using image patches
• Natural Image Illuminant Estimation via Deep Non-negative Matrix Factorisation (IET-IP, 2017, OceanUofChina)

Denoising

• Image denoising with multi-layer perceptrons (arXiv, 2012)
  • This is the very first paper using NN to image denoising tasks.
• Can a Single Image Denoising Neural Network Handle All Levels of Gaussian Noise? (SPL, 2014, France)
  • This paper proposal a way to apply NN on image denoising with different noise level of Gaussian noise.
• Deep convolutional architecture for natural image denoising (WCSP, 2015, ZJU)
• Learning Deep CNN Denoiser Prior for Image Restoration (arXiv, 2017, HKPolyU)
• Beyond a Gaussian Denoiser: Residual Learning of Deep CNN for Image Denoising (TIP, 2017, HKPolyU)
• FFDNet: Toward a Fast and Flexible Solution for CNN based Image Denoising (arXiv, 2017, HKPolyU)
• Image Restoration: From Sparse and Low-Rank Priors to Deep Priors (IPM, 2017, HKPolyU)
  • This is a review paper.
• Dilated Deep Residual Network for Image Denoising (arXiv, 2017, SIU)
• IDEAL: Image DEnoising AcceLerator (ACM, 2017, UToronto&Algolux)
  • A NN based approximations of BM3D on an NN accelerator

Demosaicing

• Demosaicing using artificial neural networks (SPIE, 2000)
• A multilayer neural network for image demosaicking (ICIP, 2012)
• Deep Joint Demosaicking and Denoising (Siggraph, 2016, MIT&Adobe)
  • This paper propose to use more difficult patches for the training.
  • Data and code are available on Github.

Automatic adjustment

From the publications, we can find that Adobe has done a lot of work pushing the usage of machine learning in low-level image processing especially automatic photo adjustment.

• Learning Photographic Global Tonal Adjustment with a Database of Input / Output Image Pairs (CVPR, 2011, MIT&Adobe)
  • Adjustment personalization
• Automatic Photo Adjustment Using Deep Neural Networks (ACM, 2016, UIUC&Adobe&Microsoft)
  • This technique is more accurate and supports local edits.

Superresolution

Super-resolution is one of the areas that NN has been applied extensively and achieved great success.

• Deep Networks for Image Super-Resolution with Sparse Prior (ICCV, 2015, UIUC)
• Image superresolution using deep convolutional networks (TPAMI, 2015, HKUST)
• Accurate image super-resolution using very deep convolutional networks (CVPR, 2016, Seoul Univ.)
• Real-time single image and video super-resolution using an efficient sub-pixel convolutional neural network (CVPR, 2016, ICL)
  • An order of magnitude faster than previous CNN-based methods
• Image Super-Resolution via Deep Recursive Residual Network (CVPR, 2017, MSU)
• Photo-realistic single image super-resolution using a generative adversarial network (CVPR, 2017, Twitter)

Artefacts removal

• Deep Generative Adversarial Compression Artifact Removal (arXiv, 2017, UoFlorence)
• Real-time Deep Video Deinterlacing (arXiv, 2017, CUHK)

Pipeline

• Learning the image processing pipeline (arXiv, 2016, Stanford)
  • Propose to learn the filter parameters by ML
• Learning Adaptive Parameter Tuning for Image Processing (arXiv, 2016, UCLA&Nvidia)

Image quality evaluation

• Deep Learning for Blind Image Quality Assessment (2017, Unimib)

Others

• Fast Image Processing with Fully-Convolutional Networks (ICCV, 2017, Intel)
  • Operator approximation to accelerate image processing tasks
• Learning Proximal Operators:Using Denoising Networks for Regularizing Inverse Imaging Problems (arXiv, 2017, TUM)
  • This paper proposes a way to use NN for the the general fidelity plus regularization optimization problem which may find applications in many problems.

• Quick Sort

TODO:

• Add descriptions of quick sort algorithm

Quick Sort

// Quick sort
void exch(int arr[], int i, int j)
{
	int temp = arr[i];
	arr[i] = arr[j];
	arr[j] = temp;
}


int partition(int arr[], int left, int right)
{
	// partition
    int i = left;
    int j = right + 1;
    int pivot = arr[left];

    while (true) 
    {
    	// scan from left to right
    	while (arr[++i] < pivot)
    		if (i == right) break;
    	// scan from right to left
    	while (arr[--j] > pivot)
    		if (j == left) break;
    	if (i > = j) break;
    	exch(arr, i, j);
    }

    exch(arr, left, j);
    return j;
}	
    		
void quickSort(int arr[], int left, int right)
{
	if (left >= right) return;
	int j = partition(arr, left, right); // partition
	quickSort(arr, left, j - 1); // sort left part a[left,...,j -1]
	quickSort(arr, j + 1, right); // sort right part a[j+1,...,right]
}

void sort(int arr[], const int len)
{
	shuffle(arr); // random shuffle the input
	quickSort(arr, 0, len - 1);
}

• Merge Sort

TODO:

• Add descriptions of merge sort algorithm
• Add C++ implementation

Merge Sort

// merge two sorted array
void mergearray(int a[], int n, int b[], int m, int c[])
{
    int i = 0;
    int j = 0;
    int k = 0;

    while (i < n && j < m)
    {
        if (a[i] < b[j])
            c[k++] = a[i++];
        else
            c[k++] = b[j++];
    }
    while (i < n)
    	c[k++] = a[i++];
    while (j < m)
    	c[k++] = b[j++];
}

The space complexity is O(N), and the time complexity is also O(N).

Top-down recursive based method

// Top-down recursive based method  

void merge_sort(int arr[], const int len)
{
	int aux[len];
	merge_sort_recursive(int arr[], int aux[], 0, len - 1);
}

void merge_sort_recursive(int arr[], int aux[], int begin, int end)
{
	int len = end - begin + 1;
	int begin1 = begin;
	int middle = begin1 + (len>>1) -1;
	int end1 = middle;
	int start2 = middle + 1;
	int end2 = end;
	merge_sort_recursive(int a[], int aux[], int begin1, int end1);
	merge_sort_recursive(int a[], int aux[], int begin2, int end2);
	int k = begin;
	while (begin1 <= end1 && begin2 <= end2)
		aux[k++] = arr[begin1] < arr[begin2] ? arr[begin1++] : arr[begin2++];
	while (begin1 <= end1)
		aux[k++] = arr[begin1++];
	while (begin2 <= end2)
		aux[k++] = arr[begin2++];
	for (k = begin; k <= end; k++)
		arr[k] = aux[k];
}

Bottom-up iterative based method

// Bottom-up iterative based method

void merge_sort_iterative(int arr[], const int len)
{
	int aux = new int[len];
	for (int seq = 1; seq < len; seq += seq)
	{
		for (int begin = 0; begin < len; begin + = seq + seq)
		{
			int begin1 = begin;
			int middle = min(begin0 + seq, len);
			int end1 = middle - 1;
			int begin2 = middle;
			int end2 = begin1 + seq + seq -1; 
			int k = begin;
			while (begin1 < end1 && begin2 < end2)
				aux[k++] = arr[begin1] < arr[begin2] ? arr[begin1++] : arr[begin2++]; 
			while (begin1 < end1)
				aux[k++] = arr[begin1++];
			while (begin2 < end2)
				aux[k++] = arr[begin2++];
		}
		for (int j = 0; j < len; j++)
			arr[j] = aux[j];
	}
}