Draft:Kihwan Kim

Kihwan Kim
File:To be added Kihwan kim at XR Unlocked (New York; 2024)
Born	(1975-03-07) 7 March 1975 Seoul, Republic of Korea
Alma mater	Yonsei University (B.S.) Georgia Institute of Technology (Ph.D.)
Title	Corporate Executive Vice President at Samsung Electronics (MX)
Scientific career
Fields	Computer Vision, Graphics, Machine learning, On-device AI, 3D Vision, XR
Korean name
Hangul	김기환
Hanja	金基煥

Kihwan Kim (Korean: 김기환; born 7 March 1975) is a computer scientist and currently the Corporate Executive Vice President of Samsung Electronics in Mobile eXperience(MX) division, with over 20 years of R&D expertise in Computer Vision, Graphics and Machine Learning. He specializes in On-Device AI, Multi-modal AI, 3D Vision, Camera pipelines and XR platforms for Samsung’s flagship Galaxy devices, such as foldable phones and VST(Visual see-through)/AR Glasses, including recently revealed Project Moohan on Android XR.

With a proven track record in research and commercialization, Kihwan has successfully led cross-functional teams to achieve mass product deployment, delivering over 10 million units annually across phones, watches and diverse mobile devices. His works also extend to XR platforms, self-driving cars and robotics.

Kihwan holds a Ph.D and M.S in Computer Science from the Georgia Institute of Technology and a B.S in Electrical Engineering from Yonsei University.

Recent highlights

XR Unlocked (Dec 2024)

On December 12 2024, Samsung and Google together unveiled a new operating system for XR, called Android XR, at the Google's developer event, "XR Unlocked", at New York City. At the event, two prototype devices have been showcased, one of which was Samsung's mixed reality (MR) headset, "Project Moohan".

Project Moohan, similar to devices like the Meta Quest and Apple Vision Pro, is a headset designed to enable immersive experiences. At the event, the device has been highlighted to seamlessly enable Google’s AI assistant, Gemini, allowing users to issue voice commands.

At an interview with Wired, Kihwan emphasized the revolutionary nature of the Android XR project where he described it as “not something that could be created by a single team or company, but a project that required an entirely different level of collaboration”.

To build the new Android XR platform, we set off on a collaboration at a level that has never been seen before between industry leaders, with experts from computer vision, graphics, machine learnings, all working together to co-design, test, and create a new operating system from the ground up.

During his keynote talk, Kihwan introduced the three core values of the Android XR platform, which underpin the vision for Project Moohan. He explained that the platform aims to create meaningful changes in everyday life by delivering an immersive visual experience through unlimited visual content, enabling natural interactions through audio-visual elements and movement, and fostering open collaboration with the Andriod, OpenXR, VR and mobile AR communities.

We believe that Android XR with Moohan opens up an entirely new way to bring AI into your life and interact with the digital and physical world.

SDC 2022 (Oct 2022)

During SDC 2022, Kihwan announced the new Avatar SDK (AR emoji SDK 2022) for Phones, Watch, TVs, and showcased the latest update of Galaxy avatar in Samsung Galaxy ecosystem.

Engineering Careers

Samsung Electronics, SDS (2001-2005)

Kihwan’s professional journey began at Samsung SDS, where he worked as a Senior Research Engineer, focusing on foundational R&D in IT systems and technology development. The major project he led:

Face Recognition - Real-time collaboration system

Real-time Collaboration System where he was responsible for face detection part. Fisher-face, and statistical skin segmentation were used for ViaFaceTM. The work has appeared in COMDEX 2001 Las Vegas. Throughout the project, he also designed/developed real-time collaboration system called Syncbiz, embedded framework for IP set-top box called LivingWise and Ubiquitous home network framework called NEX.

Disney Research (2009-2009)

In 2009, he joined Disney Research as a Research Associate, where he engaged in pioneering projects in computer vision and graphics.

Scene Analysis and Micro-casting

He conducted a project for detecting important location in the game where he designed and implemented proto-type system for micro-casting. [CVPR10A] Throughout the project, he also implemented player tracking algorithm using particle filter and mean-shift , and team classification algorithm for sports visualization. [CVPR10B]

NVIDIA (2012-2020)

From 2012 to 2020, Kihwan worked as a Principal Research Scientist at NVIDIA Research, concentrating on 3D computer vision and scene understanding for applications in robotics, AR/VR and autonomous vehicles.

Learning and Perception Research Team

He led 3D Computer vision group for inventing algorithms for autonomous driving SDK, robotics SDK, and content creation framework. [CVPR18a,b][ECCV18a,b], [CVPR19a,b,c,d], [WACV20][CVPR20a,b,c] Main projects he led include:

• Sensor-based localization project for autonomous driving: mapping and registration of points cloud captured from a Lidar sensor[3DV15][ECCV18b].
• VirtualEye (DARPA) project: 3D mapping and free view-point video.
• NVIDIA SLAM (NVSLAM): An umbrella project of a various 3D Vision projects for next generation augmented/virtual reality and autonomous driving [3DV14/15][EGSR15] [CVPR16a][ICCV17a][ICCV17b][CVPR18][ECCV18a][ARX18a][ARXV19a,b,c,d]

Some tech-transfer for NVIDIA products he contributed include:

• ISAAC SDK (Robotics)
• SDKs for DrivePX
• legacy VisionWorks
• Internal VR/AR SDKs and various open-source projects

Mobile Visual Computing Team

He led mobile visual computing team where he majorly led collaboration with Google/ATAP for Tango project (Peanut/Yellowstone). He also conducted a project for driver’s gesture recognition system for Advanced Driver Assistant System (ADAS) using multi-modal sensors and Deep Neural Network (CNN). [FG15][RADAR15][P15-a,b][IV16][CVPRW15][CVPR16b]

Some other projects he led during that time include:

• tracking and scene reconstruction research for ADAS and autonomous driving project (Sensor fusion: depth-camera, vision, IMU, etc.)
• an Real-time viewfinder editing project [SIGGRAPHA13]
• a tutorial on OpenCV for native Android: SIGGRAPH13 (mobile)
• fast Image registration and tracking for mobile vision [P14]
• Stochastic Motion field analysis using Gaussian Process [CVPR12]

Samsung Electronics, Mobile eXperience (MX) (2020-)

Kihwan returned to Samsung Electronics in 2020 as a Corporate Vice President, spearheading various camera and computer vision solutions for Galaxy flagship mobile models. Some major projects he led during this time include:

• DL-based scene recognition solutions for Galaxy phones: Bixby Vision, Single take, Relighting, Video summarization, and AI Filters [WACV22].
• DL-based UDC (Under display camera) imaging SW [STC Winner 20]
• DL-based imaging pipeline for flagship Galaxy phones: SR/NR/HDR, Night-mode, Night portrait, and Nightography [STC Winner 21]
• DL-based image synthesis for flagship Galaxy phones: Photo remaster.
• On-device AI taskforce; GPU/DSP/NPU deployment and optimization with vendor specific SDKs (EDEN, SNPE) for Galaxy devices.

In parallel, he also led Graphics R&D group where he was responsible for GPU, Graphics and rendering system optimization. Some projects he managed here include:

• Android rendering framework, Variational Refresh Rate (VRR) (Since S21)
• Game engine, and GPU optimization; adaptive perf, RL-based optimization (SPA)
• Flagship models; Galaxy S20, S21, S22, Note 20, Z Flip 2, Galaxy Z Fold 2, Galaxy Z Fold 3. A/M series, Watch 4, and S6/S7/S8 Tablets etc.

His expertise led him to become the head of the joint venture team behind VST/AR glasses involving Google and Qualcomm in 2021. Managing cross-functional worldwide teams spanning verticals, he led XR collaborations on both software and hardware innovations. He was promoted to an Executive Vice President in Samsung leading both Software and Hardware team in 2024. Some projects he has been leading in this domain include:

• XR/AR and multi-modal AI and computer vision
• XR and Avatar SW
• Lead executive of Samsung/Google/Qualcomm joint project on XR/AR- managing cross-functional world-wide teams across vertically integrated topics on BSP, System SW, perception, rendering, AI, and newly developed XR/AR specific OS

and its applications.

• Avatar commercialization (AR Emoji and applications) for Phones, watch, tablets and TV. Unity-based Avatar SDK for various Samsung services (Knox/Cloud).
• DL-based avatar reenactment commercialization, and R&D for neural rendering
• XR platform and system SW for various form factors
• On-device 3D Computer vision solutions (SLAM, Hand/Eye/Face tracking, etc.)
• OpenXR and rendering framework for various user scenarios
• Mobile AR applications (AR Doodle, Bixby Vision, AR Canvas, and AR Emoji Camera) for Galaxy phones, Tablets and watches.

Selected publications

List of selected publications:

• [WACV22] A Abdelhamed, J Yim, A Punnappurath, MS Brown, J Choe, K Kim, Extracting Vignetting and Grain Filter Effects From Photos, In Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision, WACV 2022
• [NeurIPS20] X. Li, S. Liu, K.Kim, S. De melo, X. Wang, M. Yang, J. Kautz, Online Adaptation for Consistent Mesh Reconstruction in the Wild, In Proceeding of 2020 Conference on Neural Information Processing Systems, NeurIPS 2020
• [ECCV20a] W. Yuan, B. Eckart, K.Kim, V. Jampani, D. Fox, J. Kautz, DeepGMR: Learning Latent Gaussian Mixture Models for Registration, In Proceeding of 2020 European Conference on Computer Vision, ECCV 2020
• [ECCV20b] X. Li, S. Liu, K.Kim, S. De melo, V. Jampani, M. Yang, J. Kautz, Self-supervised Single-view 3D Reconstruction via Semantic Consistency, In Proceeding of 2018 European Conference on Computer Vision, ECCV 2020
• [CVPR20a] J. Yoon, K. Kim, O. Gallo, H. Park, J. Kautz, Novel View Synthesis of Dynamic Scenes with Globally Coherent Depths, In Proceeding of IEEE Conference on Computer Vision and Pattern Recognition, CVPR 2020
• [CVPR20b] M. Boss, V. Jampani, K. Kim, H. Lensch, J. Kautz, Two-shot Spatially varying BRDF and Shape Estimation, In Proceeding of IEEE Conference on Computer Vision and Pattern Recognition, CVPR 2020
• [CVPR20c] A. Badki, O. Gallo, A. Troccoli, K. Kim, P. Sen, J. Kautz, Bi3D: Stereo Depth Estimation via Binary Classifications, In Proceeding of IEEE Conference on Computer Vision and Pattern Recognition, CVPR 2020
• [WACV20] M. Innmann, K.Kim, J. Gu, M. Nießner, C. Loop, M. Stamminger, J.Kautz, NR-MVS: Non-Rigid Multi-view Stereo. In Proceeding of 2020 IEEE Winter Conference on Applications on Computer Vision, WACV 2020 (Oral)
• [ICCV19] S.Sengupta, J. Gu, K.Kim, G.Liu, D. Jacobs, J.Kautz, Neural Inverse Rendering of an Indoor Scene from a Single Image. In Proceeding of 2019 IEEE International Conference on Computer Vision, ICCV 2019
• [CVPR19a] C.Liu, K.Kim, J.Gu, Y. Furukawa, J.Kautz, PlaneRCNN: 3D Plane Detection and Reconstruction from a Single View, In Proceeding of IEEE Conference on Computer Vision and Pattern Recognition, CVPR 2019 (Oral)
• [CVPR19b] C.Liu, J.Gu, K.Kim, S. Narasimhan, J.Kautz, Neural RGB→D Sensing: Depth and Uncertainty from a Video Camera, In Proceeding of IEEE Conference on Computer Vision and Pattern Recognition, CVPR 2019. (Oral) *Best paper finalist
• [CVPR19c] A. Ranjan, V. Jampani, K.Kim, D.Sun, L.Balles, J.Wulff, M.Black, Competitive Collaboration: Joint Unsupervised Learning of Depth, Camera Motion, Optical Flow and Motion Segmentation. In Proceeding of 2019 IEEE Conference on Computer Vision and Pattern Recognition, CVPR 2019
• [CVPR19d] X. Li, S. Liu, K.Kim, X. Wang, M. Wang, J. Kautz, Putting Humans in a Scene: Learning Affordance in 3D Indoor Environments. In Proceeding of 2019 IEEE Conference on Computer Vision and Pattern Recognition, CVPR 2019
• [ECCV18a] Z. Lv, K. Kim, A. Troccoli, D. Sun, J. Rehg, J. Kautz, Learning Rigidity in Dynamic Scenes with a Moving Camera for 3D Motion Field Estimation , In Proceeding of 2018 European Conference on Computer Vision, ECCV 2018
• [ECCV18b] B. Eckart, K. Kim, J. Kautz, Fast and Accurate Point Cloud Registration using Trees of Gaussian Mixtures , In Proceeding of 2018 European Conference on Computer Vision, ECCV 2018
• [CVPR18] S. Brahmbhatt, J. Gu, K. Kim, J. Hays, J. Kautz, Geometry-Aware Learning of Maps for Camera Localization (MapNet) , In Proceeding of 2018 IEEE Conference on Computer Vision and Pattern Recognition, CVPR 2018 (Oral)
• [ICCV17a] K. Kim, J. Gu, S. Tyree, P. Molchanov, M. Nießner, J. Kautz, A Lightweight Approach for On-the-Fly Reflectance Estimation, In Proceeding of 2017 IEEE International Conference on Computer Vision, ICCV 2017 (Oral)
• [ICCV17b] R. Maier, K. Kim, M. Nießner, D. Cremers, J. Kautz, Intrinsic3D: High-Quality 3D Reconstruction by Joint Appearance and Geometry Optimization with Spatially-Varying Lighting, In Proceeding of 2017 IEEE International Conference on Computer Vision, ICCV 2017
• [3DV17] V. Golyanik, K. Kim, R. Maier, M. Nießner, J. Kautz, Multiframe Scene Flow with Piecewise Rigid Motion, In Proceeding of 2017 IEEE International Conference on 3D Vision, 3DV 2017 (Oral)
• [CVPR16a] B. Eckart, K. Kim, A. Troccoli, A. Kelly, J. Kautz, Accelerated Generative Models for 3D Point Cloud Data, In Proceeding of 2016 IEEE Conference on Computer Vision and Pattern Recognition, CVPR 2016 (Oral)
• [CVPR16b] P. Molchanov, X. Yang, S. Gupta, K. Kim, S. Tyree, J. Kautz, Online Detection and Classification of Dynamic Hand Gestures with Recurrent 3D Convolutional Neural Networks, In Proceeding of 2016 IEEE Conference on Computer Vision and Pattern Recognition, CVPR 2016
• [IV16] S. Gupta, P. Molchanov, X. Yang, K. Kim, S. Tyree, J. Kautz, Towards Selecting Robust Hand Gestures for Automotive Interfaces, In Proceeding of 2016 IEEE Intelligent Vehicles Symposium, IV 2016 (Oral)
• [3DV15] B. Eckart, K. Kim, A. Troccoli, A. Kelly, J. Kautz, MLMD: Maximum Likelihood Mixture Decoupling for Fast and Accurate Point Cloud Registration, In IEEE 3D Vision, 3DV2015 (Oral)
• [EGSR15] S. U. Mehta, K. Kim, D. Pajak, K. Pulli, J. Kautz, R. Ramamoorthi, Filtering Environment Illumination for Interactive Physically-Based Rendering in Mixed Reality, In Eurographics Symposium on Rendering, EGSR 2015 (Oral)
• [CVPRW15] P. Molchanov, S. Gupta, K. Kim, J. Kautz, Hand Gesture Recognition with 3D Convolutional Neural Networks, In IEEE CVPR 2015 Workshop on Hand Gesture Recognition
• [FG15] P. Molchanov, S. Gupta, K. Kim, K. Pulli, Multi-sensor System for Driver’s Hand-Gesture Recognition, In IEEE Automatic Face and Gesture recognition, FG2015 (Oral)
• [RADAR15] P. Molchanov, S. Gupta, K. Kim, K. Pulli, Short-Range FMCW Monopulse Radar for Hand-Gesture Sensing, In IEEE International Radar Conference 2015
• [3DV14] D. Herrera, K. Kim, J. Kannala, K. Pulli, and J. Heikkila , DT-SLAM: Deferred Triangulation for Robust SLAM, In IEEE 3D Vision, 3DV2015
• [SIGGRAPHA13] J. Baek, D. Pajak, K. Kim, K. Pulli, and M. Levoy, WYSIWYG Computational Photography via Viewfinder Editing, In ACM Transactions on Graphics, SIGGRAPH Asia 2013
• [CVPR12] K. Kim, D. Lee, and I. Essa, Detecting Regions of Interest in Dynamic Scenes with Camera Motions, In Proceeding of 2012 IEEE Conference on Computer Vision and Pattern Recognition
• [ICCV11] K. Kim, D. Lee, and I. Essa, Gaussian Process Regression Flow for Analysis of Motion Trajectories, In Proceeding of 2011 IEEE International Conference on Computer Vision
• [CVPR10a] K. Kim, M. Grundmann, A. Shamir, I. Matthews, J. Hodgins, and I. Essa, Motion Fields to Predict Play Evolution in Dynamic Sports Scenes, In Proceeding of 2010 IEEE Conference on Computer Vision and Pattern Recognition
• [CVPR10b] R. Hamid, R. Kumar, M. Grundmann, K. Kim, I. Essa and J. Hodgins, Player Localization Using Multiple Static Cameras for Sports Visualization, In Proceeding of 2010 IEEE Conference on Computer Vision and Pattern Recognition
• [ISMAR09] K. Kim, S. Oh, J. Lee and I. Essa, Augmenting Aerial Earth Maps with Dynamic Information, In Proceeding of 2009 IEEE/ACM International Symposium on Mixed and Augmented Reality
• [ISWC08] K. Kim, J. Summet, T. Starner, D. Ashbrook, M. Kapade and I. Essa, Localization and 3D Reconstruction of Urban Scenes Using GPS, In Proceeding of 2008 IEEE International Conference on Wearable Computers
• [ACMMM06] K. Kim, I. Essa and G. D. Abowd, Interactive Mosaic Generation for Video Navigation, In Proceeding of 2006 ACM International Conference on Multimedia
• [TOG13] J. Baek, D. Pajak, K. Kim, K. Pulli, and M. Levoy, WYSIWYG Computational Photography via Viewfinder Editing, In ACM Transactions on Graphics, Volume 32.
• [VR11] K. Kim, S. Oh, J. Lee and I. Essa, Augmenting Aerial Earthmaps with Dynamic Information from Videos , In Virtual Reality Journal [Special issue on Augmented Reality], Springer London, 2011 (VR)
• [JGT08] B. Kim, K. Kim and G. Turk, A Shadow Volume Algorithm for Opaque and Transparent Non-Manifold Casters, In Journal of Graphics Tools, A.K. Peters, 2008