Print

发布时间: 2017-05-16
摘要点击次数:
全文下载次数:
DOI: 10.11834/jig.160591
2017 | Volume 22 | Number 5




    第十八届全国图像图形
学术会议专栏    




  <<上一篇 




  下一篇>> 





分频能量调整在高分辨傅里叶显微技术中的应用
expand article info 旷雅唯1,2, 段侪杰1,2, 马辉1,3
1. 深圳市无损和微创医疗技术重点实验室, 清华大学深圳研究生院, 深圳 518055;
2. 清华大学生物医学工程系, 北京 100084;
3. 清华大学物理系, 北京 100084

摘要

目的 高分辨傅里叶显微技术(FPM)是利用一组不同角度入射光下采集的低分辨率图像重建高分辨率图像的技术,该技术主要的理论基础是相位还原和综合孔径技术。低分辨图像和高分辨率图像在频域中的差异体现在高频段中的能量,高分辨率图像高频段能量更多。但是此前的方法重建的图像在高频段内的能量仍然较少。针对该问题,提出了一种新的FPM迭代更新模式——分频能量调整(BE)。 方法 基于高分辨率图像在傅里叶空间的能量分布的先验,在迭代过程中加入分频能量调整,来约束更新过程中的能量分布,从而使重建图像在能量上更接近于高分辨率图像,进一步提高图像的分辨率,突出边缘信息。 结果 在光学分辨率检验板和蚕豆气孔数据上对比增加光瞳函数恢复的FPM方法(EPRY-FPM)和添加分频能量调整的FPM方法(BE-FPM),实验表明,BE-FPM能进一步提高重建图像分辨率,突出边缘信息。为验证算法的鲁棒性,对样本添加模拟产生的高斯噪声和椒盐噪声,重建结果的视觉效果表明本文方法对噪声的鲁棒性更优。 结论 本文方法能进一步提高重建图像的分辨率,并且突出边缘信息。在噪声图像中比EPRY-FPM的更新模式具有更高的鲁棒性。在生物样本中,很多的图像具有相似的分布,而相似分布的样本在傅里叶空间的能量分布具有一致性,因此,BE-FPM方法在部分高分辨率样本重建大样本,单幅高分辨率样本重建同类样本等问题上有较大的应用潜力。

关键词

高分辨傅里叶显微技术; 高分辨率; 图像重建; 能量分布; 傅里叶空间

Application of band energy adjustment in Fourier ptychographic microscopy
expand article info Kuang Yawei1,2, Duan Chaijie1,2, Ma Hui1,3
1. Shenzhen Key Laboratory for Nondestructive and Minimal Invasive Medical Technologies, Graduate School at Shenzhen, Tsinghua University, Shenzhen 518055, China;
2. Department of Biomedical Engineering, Tsinghua University, Beijing 100084, China;
3. Department of Physics, Tsinghua University, Beijing 100084, China
Supported by: National Natural Science Foundation of China (81230035, 61527826)

Abstract

Objective Fourierptychographic microscopy (FPM) is an imaging technique for reconstructing high-resolution images using low-resolution images acquired from a set of different angles of incident light. This technique can bypass the resolution limit of employed optics. The FPM algorithm comprises two main theoretical bases. The first one is the phase retrieval technique, which was originally developed for electron imaging. This technique is used to recover the lost phase information using intensity measurements, and it typically consists of alternating enforcement of the known information of the object in the spatial and Fourier domains. The second one is the aperture synthesis. This technique was originally deve-loped for radio astronomy to pass the resolution limit of the single radio telescope. The basic idea of this technique is to combine images from a collection of telescopes in the Fourier domain to improve the resolution. By integrating the two techniques, the FPM can transform a conventional microscope into a high-resolution, wide field-of-view one. The difference between the low-resolution image and the high-resolution image in the frequency domain is reflected in the energy in the high-frequency band, and the high-frequency energy is abundant in the high-resolution image. However, the energy in the high-frequency band reconstructed by the former algorithm remains small. This study proposes a new iterative updating mode of FPM-band energy adjustment in FPM (BE-FPM) to solve the problem. Method This method is based on the energy distribution of Fourier space in high-resolution images. The entire iteration process for every image is divided into two steps. The first step conducts the recovery depending on the concepts of conventional FPM, which is to update the sub-region of the Fourier spectrum by the recorded low-resolution images. The second step is to use the new updating mode, namely, band energy adjustment in the iterative process. Energy distribution of a high-resolution image, which is calculated from a similar high-resolution sample, is applied as the prior. The Fourier spectrum is divided into several bands. Every band has different frequency ranges. The energy of each band is calculated and adjusted by the high-resolution prior. The reconstructed image is brought closer to the high-resolution image by adjusting the energy of different frequency bands. After the iterative process for one image, the process is conducted for every captured low-resolution image several times until the convergence is achieved. Experimental results on resolution board and bean hole data demonstrate that the BE-FPM further improves the resolution of the reconstructed image and can highlight the edge information. Result We conduct the experiments on resolution board and bean hole data. Compared with the updating mode used in embedded pupil function recovery for FPM (EPRY-FPM) and the BE-FPM updating mode, the BE-FPM mode can further improve the resolution of the reconstructed image and highlight the edge information. The element of group eight in the resolution board has a better and clearer reconstruction effect in the BE-FPM reconstruction result. The boundary of bean hole achieves a much clearer reconstruction by using the BE-FPM. Gaussian noise and salt-and-pepper noise are added to the originally captured low-resolution images to prove the robustness of the BE-FPM. A reconstruction of the noisy images using the EPRY-FPM and BE-FPM proves that the robustness of the BE-FPM for noise is better than that of the EPRY-FPM. Conclusion This paper presents a new iterative updating mode of FPM, namely, BE-FPM. Experiments on resolution board and bean hole data show that the BE-FPM updating mode can further improve the resolution of the reconstructed image and highlight the edge information. The BE-FPM updating mode is more robust than the EPRY-FPM when the recorded images contain noise. In biological samples, numerous images have similar distributions, and these samples have similar energy distributions in the Fourier space. Therefore, the BE-FPM has potentials in reconstructing an entire sample using a partial high-resolution image and reconstructing samples in the same class via a single high-resolution image.

Key words

Fourier ptychographic microscopy; high resolution; image reconstruction techniques; energy distribution; Fourier space

0 引言

高分辨傅里叶显微技术 (FPM)[1]是一种最新发展的超分辨率显微成像技术,2013年首次在Nature Photonics上提出。利用傅里叶光学的理论,将普通光学显微镜的柯勒照明方式改为由多个LED点光源照明的方式,并假设在观测样本的照明角度范围内LED发出的是准相干平面波。通过依次点亮LED点光源,采用多个角度照明的方法,利用低倍物镜采集不同入射角度下的图像。通过对低倍镜图像序列利用相位恢复算法,迭代重建出单张的高分辨率图像[2-7]。该方法本质上是通过改变光源入射角度,获得样品传递函数在傅里叶空间中的不同频率域的信息。通过大角度照明,获取函数的高频信息,起到了增大了显微镜物镜的数值孔径的效果,能够在保留低倍镜广视场优势的同时提高图像分辨率,在医疗,遥感等领域有很大的应用价值。

目前FPM技术研究主要有两个方向:一个是对整个采集和重建过程的加速[8-11],例如,通过优化光源的排列方式来减少图像的采集数量,或改变点亮模式,如随机点亮多个LED光源进行成像来实现加速;另一个方向是对图像重建质量的提高[2, 4, 12],Ou等人[2]提出了增加光瞳函数恢复的FPM算法 (EPRY-FPM) 不仅能提高图像重建质量,而且能让这个重建过程更加稳定、高效。Bian等人[4]基于样本空间频谱的不均匀性提出了自适应FPM算法 (AFP),该方法只利用了一部分的光源,自适应地从中心低频到周边高频中选择高分辨率频谱中最重要的频段成像。

综上所述,目前FPM已经有了广泛的改进,但是对于频谱能量的利用,算法对噪声的鲁棒性等方面仍有一定的改良空间。本文旨在利用傅里叶频谱能量分布提出一个新的迭代更新方式——BE-FPM,从而进一步提高重建分辨率,突出边缘信息,加强算法对噪声的鲁棒性。

1 理论基础

1.1 FPM理论基础

FPM算法分图像采集和重建两个过程,采集平台示意图如图 1所示。

图 1 FPM采集示意图
Fig. 1 Acquisition schematic

依次点亮LED,获得低分辨率图像序列。其具体过程如下:

光源发出的入射平面波通过样本后,振幅和相位会发生变化。记空域位置$\boldsymbol{r}=\left( x, y \right)$,频域位置为$\boldsymbol{u}=\left( {{k}_{x}}, {{k}_{y}} \right)$,将第$n$个光源发出的光波记为${{\boldsymbol{U}}_{n}}=\left( {{k}_{xn}}, {{k}_{vn}} \right)$,样本的复传递函数记为$\boldsymbol{s}\left( \boldsymbol{r} \right)$,那么通过样本之后的出射光为

${{\boldsymbol{e}}_{n}}\left( \boldsymbol{r} \right)=\boldsymbol{s}\left( \boldsymbol{r} \right){{\text{e}}^{\text{i}{{k}_{xn}}\cdot x+\text{i}{{k}_{yn}}\cdot y}}$ (1)

式中,${{\text{e}}^{\text{i}{{k}_{xn}}\cdot \text{i}{{k}_{yn}}\cdot y}}$表示光波${{\boldsymbol{U}}_{n}}$下的入射平面波,i为虚数单位。出射光通过物镜,与物镜的点扩散函数$\boldsymbol{p}\left( \boldsymbol{r} \right)$卷积,到达CCD平面前的光波为

${\boldsymbol{\phi} _n}\left( \boldsymbol{r} \right) = {\boldsymbol{e}_n}\left( \boldsymbol{r} \right) \otimes \boldsymbol{p}\left( \boldsymbol{r} \right)$ (2)

CCD采集的强度,即获得的强度图像表示为

${\boldsymbol{I}_n} = {\left| {{\boldsymbol{\phi} _n}\left( \boldsymbol{r} \right)} \right|^2} = {\left| {{\boldsymbol{e}_n}\left( \boldsymbol{r} \right) \otimes \boldsymbol{p}\left( \boldsymbol{r} \right)} \right|^2}$ (3)

将样本传递函数和点扩散函数进行傅里叶变换,即

$\boldsymbol{P}\left( \boldsymbol{u} \right) = F\left[{\boldsymbol{p}\left( \boldsymbol{r} \right)} \right]$ (4)

$\boldsymbol{S}\left( \boldsymbol{u} \right) = F\left[{\boldsymbol{s}\left( \boldsymbol{r} \right)} \right]$ (5)

式中,$F\left[\cdot \right]$指傅里叶变换。那么出射光的傅里叶变换为

$F\left[{{\boldsymbol{e}_n}\left( \boldsymbol{r} \right)} \right] = \boldsymbol{S}\left( {\boldsymbol{u} -{\boldsymbol{U}_\boldsymbol{n}}} \right)$ (6)

从而,CCD采集强度也可以表示为

${\boldsymbol{I}_\boldsymbol{n}} = {\left| {{F^{- 1}}\left[{\boldsymbol{S}\left( {\boldsymbol{u}-{\boldsymbol{U}_\boldsymbol{n}}} \right) \cdot \boldsymbol{P}\left( \boldsymbol{u} \right)} \right]} \right|^2}$ (7)

可见,不同角度的准相干平面波入射,等效于在傅里叶空间将样本参数做平移。

在此基础上,选定中心图像作为重建起始图像,通过采集参数估算其傅里叶空间,然后通过基于迭代的相位还原方法,利用采集的低分辨率图像的傅里叶空间填充高分辨率图像的傅里叶空间,实现高频信息的重建,从而得到高分辨率图像。

1.2 分频能量调整理论基础

高分辨率图像和低分辨率图像在频域上的能量分布有明显的差异,即高分辨率图像拥有更多的高频信息。从傅里叶空间来看,高分辨率图像在高频区域具有更高的能量,如图 2所示。

图 2 低分辨率图像和高分辨率图对比
Fig. 2 Comparison of low resolution and high resolution images ((a) intensity image of low-resolution; (b) intensity image of high-resolution; (c) normalized Fourier spectrum of (a) and (b), respectively)

低分辨率图像图 2(a)是在2倍镜下采集,高分辨率图像图 2(b)是在10倍物镜下采集。其傅里叶空间能量分布对比也十分明显,图 2(c)分别是图 2(a)(b)对应的归一化频谱图,可以看到,高分辨率图像在高频段信息量更多。

而低分辨率图像本身是不包含高频信息的,所以直接利用低分辨率图像来估计高频信息其准确性存在质疑,而且样本分布的方向性、噪声等会对重建结果带来较大影响。同时,具有相似分布的样本高分辨率图像的能量分布存在相似性[13],由此,设计了一种新的迭代更新方法,即分频能量调整。该方法在增加光瞳函数恢复的FPM方法 (EPRY-FPM[2]) 的基础上,利用相似样本高分辨率图像能量分布的先验知识,加入了对傅里叶空间分频能量的调整,进一步提高重建图像的分辨率。

2 分频能量调整算法

添加分频能量调整的FPM算法 (BE-FPM) 迭代的各个步骤如下:

首先利用中心光源照射下的低分辨率图像,通过升采样的方式对高分辨率图像进行初始化,并得到其傅里叶空间函数${\boldsymbol{S}_0}\left( \boldsymbol{u} \right)$。初始化光瞳函数${\boldsymbol{P}_0}\left( \boldsymbol{u} \right)$,为一个理想的圆形低通滤波器,其半径由采集平台和光学参数决定。

更新时,提取出第$n$个LED光源照明下图像的傅里叶变换函数,其对应的待重建图像傅里叶空间的频率区域为$\boldsymbol{S}\left( {\boldsymbol{u}-{\boldsymbol{U}_n}} \right)$,平移矢量${{\boldsymbol{U}_n}}$和入射光的角度有关[1]。将其与光瞳函数相乘,得到高分辨率傅里叶空间对应区域的估计函数${\mathit{\boldsymbol{ \boldsymbol{\varPhi} }}_n}\left( \boldsymbol{u} \right)$,将该区域进行傅里叶反变换,得到修正前的振幅图像${\boldsymbol{\phi} _n}\left( \boldsymbol{r} \right)$

该对应位置光源采集的强度图为${\boldsymbol{I}_{{U_n}}}\left( \boldsymbol{r} \right)$,为实际光波振幅的平方。修正后的空域复数图像为

${\boldsymbol{\phi} '_n}\left( \boldsymbol{r} \right) = \sqrt {{\boldsymbol{I}_{{U_n}}}\left( \boldsymbol{r} \right)} \frac{{{\boldsymbol{\phi} _n}\left( \boldsymbol{r} \right)}}{{\left| {{\boldsymbol{\phi} _n}\left( \boldsymbol{r} \right)} \right|}}$ (8)

使用修正后图像${\boldsymbol{\phi} '_n}\left( \boldsymbol{r} \right)$的傅里叶空间数据对重建高分辨率图像的傅里叶域进行更新,即

$\begin{array}{l} {\boldsymbol{S}_{n + 1}}\left( \boldsymbol{u} \right) = {\boldsymbol{S}_n}\left( \boldsymbol{u} \right)+\frac{{\boldsymbol{P}_0^*\left( {\boldsymbol{u} + {\boldsymbol{U}_n}} \right)}}{{\left| {{\boldsymbol{P}_0}\left( {\boldsymbol{u} + {\boldsymbol{U}_n}} \right)} \right|_{\max }^2}}\\ \left[{{{\boldsymbol{\Phi '}}_n}\left( {\boldsymbol{u} + {\boldsymbol{U}_n}} \right)-{\mathit{\boldsymbol{ \boldsymbol{\varPhi} }}_n}\left( {\boldsymbol{u} + {\boldsymbol{U}_n}} \right)} \right] \end{array}$ (9)

然后将傅里叶空间分为$m$个频段,其中$f_j$表示其第$j$个频段,其中$j \in \left[{1, 2, \cdots, m-1, m} \right]$。分别计算更新傅里叶空间和高分辨率傅里叶空间的分频能量向量

$\begin{array}{c} \boldsymbol{BE}\left( {{\boldsymbol{S}_{n + 1}}\left( \boldsymbol{u} \right)} \right) = \\ \left[{{E_{{S_{n + 1}}{f_1}}}, \cdots, {E_{{S_{n + 1}}{f_j}}}, \cdots, {E_{{S_{n + 1}}{f_m}}}} \right] \end{array}$ (10)

$\begin{array}{c} \boldsymbol{BE}\left( {\boldsymbol{HR}} \right) = \\ \left[{{E_{HR{f_1}}}, \cdots, {E_{HR{f_j}}}, \cdots, {E_{HR{f_m}}}} \right] \end{array}$ (11)

式中,$E$为频段能量。

计算调整因子$\boldsymbol{F}$

$\begin{array}{c} \boldsymbol{F = }\frac{{\boldsymbol{BE}\left( {{\boldsymbol{S}_{n + 1}}\left( \boldsymbol{u} \right)} \right)}}{{\boldsymbol{BE}\left( {\boldsymbol{HR}} \right)}} = \\ \left[{{F_{{f_1}}}, \cdots, {F_{{f_j}}}, \cdots, {F_{{f_m}}}} \right] \end{array}$ (12)

进一步更新傅里叶空间

${\boldsymbol{S'}_{n + 1}}\left( \boldsymbol{u} \right) = \left\{ \begin{array}{l} \frac{{{\boldsymbol{S}_{n + 1}}\left( \boldsymbol{u} \right)}}{{{\boldsymbol{F}_{{f_1}}}}}\;\;0 \le \boldsymbol{u} < {f_1}\\ \;\;\;\; \vdots \\ \frac{{{\boldsymbol{S}_{n + 1}}\left( \boldsymbol{u} \right)}}{{{\boldsymbol{F}_{{f_m}}}}}\;\;{f_{m-1}} \le \boldsymbol{u} < {f_m} \end{array} \right.$ (13)

至此,单幅图的更新过程完成,按照一定顺序更新完整个低分辨率图像序列之后,进入新的一轮迭代,迭代若干次,即可得到提高分辨率后的图像。

BE-FPM流程图如图 3所示。

图 3 BE-FPM迭代流程图
Fig. 3 Flowchart of BE-FPM

3 实验结果及比较

成像设备是建立在Olympus BX 43显微镜上,物镜是数值孔径 (NA) 为0.08的二倍镜,LED光源板为9×9模式,一共采集低分辨率图81幅,相邻光源间距4 mm。LED颜色为蓝色,中心波长472 nm。LED平面到样本平面距离10 cm。相机为12位黑白CCD相机,模拟像素大小3.75×3.75 μm。实验样本是光学分辨率板数据和蚕豆样本数据。结果见图 4图 5。可以看到,BE-FPM法能将分辨率进一步提高,在光学分辨率板中,BE-FPM法第8组的重建结果就优于EPRY-FPM法。从蚕豆气孔样本中,可以看出对于图像的边缘,BE-FPM能获得边缘更明显的结果。

图 4 分辨率板重建结果对比图
Fig. 4 Comparison of reconstruction results onresolution board ((a) low-resolution image with partial magnified part and the normalized Fourier spectrum acquired with 2x objective lens; (b) EPRY-FPM; (c) BE-FPM)
图 5 蚕豆重建结果对比图
Fig. 5 Comparison of reconstruction results on bean hole ((a) low-resolution image with partial magnified part and the normalized Fourier spectrum acquired with 2x objective lens; (b) EPRY-FPM; (c) BE-FPM)

为验证BE-FPM对噪声的鲁棒性,对采集的图像添加高斯噪声和椒盐噪声,然后分别用EPRY-FPM法和BE-FPM法进行重建,结果如图 6图 7所示。从图中可以明显看出BE-FPM法对含噪声图像的重建结果优于EPRY-FPM法,分辨率和边缘突出性都有较大的提升。

图 6 添加高斯噪声之后分辨率板重建结果对比图
Fig. 6 Comparison of reconstruction results on resolution board with Gaussian noise ((a) low-resolution image with partial magnified part and the normalized Fourier spectrum acquired with 2x objective lens; (b) EPRY-FPM; (c) BE-FPM)
图 7 添加椒盐噪声之后分辨率板重建结果对比图
Fig. 7 Comparison of reconstruction results on resolution board with the salt and pepper noise ((a) low-resolution image with partial magnified part and the normalized Fourier spectrum acquired with 2x objective lens; (b) EPRY-FPM; (c) BE-FPM)

以上结果中,图像亮度不一致,分析如下:在FPM重建中,采集的入射角度大的图像越多,重建的高频信息会越多,同时会使亮度更低。所以,BE-FPM重建后的高频部分能量较大,而重建结果亮度较低;EPRY-FPM法重建后高频能量较小,重建结果亮度较高。由于添加的噪声属于高频信息,EPRY-FPM重建没有对该高频信息进行处理,所以重建结果高频较多,亮度较低。但是BE-FPM法利用高分辨率图像分频能量的先验,对高频段的噪声信息做了校正处理,使高频能量减少,因此亮度会比EPRY-FPM的亮度高。

4 结论

本文设计了一种新的迭代更新方式,基于高分辨图像傅里叶空间能量分布的先验,进一步提高了图像的重建质量,突出边缘信息,增强了算法对噪声的鲁棒性。由于相似样本的高分辨率傅里叶空间分布具有相似性,一种能量分布模式可以重建多个样本,或在大样本中,获得样本的一部分高分辨率图像,从而重建整个样本。因此,该方法具有广泛的拓展意义,为FPM方法应用于大样本和利用FPM方法实现样本分类打下基础。

未来,将进一步扩展样本类型,并对图像质量的评估进行量化。

参考文献

  • [1] Zheng G A, Horstmeyer R, Yang C H. Wide-field, high-resolution Fourier ptychographic microscopy[J]. Nature Photonics, 2013, 7(9): 739–745. [DOI:10.1038/nphoton.2013.187]
  • [2] Ou X Z, Zheng G A, Yang C H. Embedded pupil function recovery for Fourier ptychographic microscopy[J]. Optics Express, 2014, 22(5): 4960–4972. [DOI:10.1364/OE.22.004960]
  • [3] Horstmeyer R, Yang C H. A phase space model of Fourier ptychographic microscopy[J]. Optics Express, 2014, 22(1): 338–358. [DOI:10.1364/OE.22.000338]
  • [4] Bian Z C, Dong S Y, Zheng G A. Adaptive system correction for robust Fourier ptychographic imaging[J]. Optics Express, 2013, 21(26): 32400–32410. [DOI:10.1364/OE.21.032400]
  • [5] Fienup J R. Phase retrieval algorithms:a comparison[J]. Applied Optics, 1982, 21(15): 2758–2769. [DOI:10.1364/AO.21.002758]
  • [6] Fienup J R. Reconstruction of a complex-valued object from the modulus of its Fourier transform using a support constraint[J]. Journal of the Optical Society of America A, 1987, 4(1): 118–123. [DOI:10.1364/JOSAA.4.000118]
  • [7] Fienup J R. Reconstruction of an object from the modulus of its Fourier transform[J]. Optics Letters, 1978, 3(1): 27–29. [DOI:10.1364/OL.3.000027]
  • [8] Guo K K, Dong S Y, Nanda P, et al. Optimization of sampling pattern and the design of Fourier ptychographic illuminator[J]. Optics Express, 2015, 23(5): 6171–6180. [DOI:10.1364/OE.23.006171]
  • [9] Dong S Y, Shiradkar R, Nanda P, et al. Spectral multiplexing and coherent-state decomposition in Fourier ptychographic imaging[J]. Biomedical Optics Express, 2014, 5(6): 1757–1767. [DOI:10.1364/BOE.5.001757]
  • [10] Dong S Y, Guo K K, Jiang S W, et al. Recovering higher dimensional image data using multiplexed structured illumination[J]. Optics Express, 2015, 23(23): 30393–30398. [DOI:10.1364/OE.23.030393]
  • [11] Tian L, Li X, Ramchandran K, et al. Multiplexed coded illumination for Fourier Ptychography with an LED array microscope[J]. Biomedical Optics Express, 2014, 5(7): 2376–2389. [DOI:10.1364/BOE.5.002376]
  • [12] Bian L H, Suo J L, Situ G H, et al. Content adaptive illumination for Fourier ptychography[J]. Optics Letters, 2014, 39(23): 6648–6651. [DOI:10.1364/OL.39.006648]
  • [13] Zhou Y, Pang Q. Plant similarity analysis based on Fourier spectrum and its angular and radialfeatures[J]. Computer Systems & Applications, 2012, 21(11): 161–164. [周游, 庞全. 傅里叶频谱径角特征的植物相似性[J]. 计算机系统应用, 2012, 21(11): 161–164. ] [DOI:10.3969/j.issn.1003-3254.2012.11.037]