Blark Runfa Lee 李润发

selected publications

1. 

   MonoPLFlowNet: Permutohedral Lattice FlowNet for Real-Scale 3D Scene Flow Estimation with Monocular Images

   Runfa Li, and Truong Nguyen

   In Proceedings of the European Conference on Computer Vision (ECCV) 2022

   Abs Bib HTML

   Real-scale scene flow estimation has become increasingly important for 3D computer vision. Some works successfully estimate real-scale 3D scene flow with LiDAR. However, these ubiquitous and expensive sensors are still unlikely to be equipped widely for real application. Other works use monocular images to estimate scene flow, but their scene flow estimations are normalized with scale ambiguity, where additional depth or point cloud ground truth are required to recover the real scale. Even though they perform well in 2D, these works do not provide accurate and reliable 3D estimates. We present a deep learning architecture on permutohedral lattice - MonoPLFlowNet. Different from all previous works, our MonoPLFlowNet is the first work where only two consecutive monocular images are used as input, while both depth and 3D scene flow are estimated in real scale. Our real-scale scene flow estimation outperforms all state-of-the-art monocular-image based works recovered to real scale by ground truth, and is comparable to LiDAR approaches. As a by-product, our real-scale depth estimation is also comparable to other state-of-the-art works.

   @inproceedings{MonoPLFlowNet,
     author = {Li, Runfa and Nguyen, Truong},
     title = {MonoPLFlowNet: Permutohedral Lattice FlowNet for Real-Scale 3D Scene Flow Estimation with Monocular Images},
     booktitle = {Proceedings of the European Conference on Computer Vision (ECCV)},
     year = {2022},
     publisher = {Springer Nature Switzerland},
     address = {Cham},
     pages = {322--339},
     doi = {10.1007/978-3-031-19812-0_19}
   }

2. 

   SM3D: Simultaneous Monocular Mapping and 3D Detection

   Runfa Li, and Truong Nguyen

   In IEEE International Conference on Image Processing (ICIP) 2021

   Abs Bib HTML

   Mapping and 3D detection are two major issues in vision-based robotics, and self-driving. While previous works only focus on each task separately, we present an innovative and efficient multi-task deep learning framework (SM3D) for Simultaneous Mapping and 3D Detection by bridging the gap with robust depth estimation and “Pseudo-Lidar” point cloud for the first time. The Mapping module takes consecutive monocular frames to generate depth and pose estimation. In 3D Detection module, the depth estimation is projected into 3D space to generate “Pseudo-Lidar” point cloud, where Lidar-based 3D detector can be leveraged on point cloud for vehicular 3D detection and localization. By end-to-end training of both modules, the proposed mapping and 3D detection method outperforms the state-of-the-art baseline by 10.0% and 13.2% in accuracy, respectively. While achieving better accuracy, our monocular multi-task SM3D is more than 2 times faster than the state of the art pure stereo 3D detector, and 18.3% faster than using two modules separately.

   @inproceedings{SM3D,
     author = {Li, Runfa and Nguyen, Truong},
     booktitle = {IEEE International Conference on Image Processing (ICIP)},
     title = {SM3D: Simultaneous Monocular Mapping and 3D Detection},
     year = {2021},
     pages = {3652-3656},
     doi = {10.1109/ICIP42928.2021.9506302},
   }