One of the most fundamental challenges in the digital age is to automatically create accurate, high-quality 3D representations of the world around us. This would have far-reaching impact, from enabling advances entertainment and immersive technologies (e.g., mixed reality) to medical applications and industrial manufacturing pipelines. Despite remarkable progress in scanning devices and 3D reconstruction algorithms, the resulting models remain highly impractical for display or use in virtual environments. This is due to the limited quality of these reconstructed 3D models, which is still far from the quality of assets designed by professional artists in countless of working hours. We believe that the key to addressing these shortcomings is understanding the design process of artist-created assets. We can then learn the correlation to real-world observations and replicate the process conditioned on these real-world input scans. In this proposal, we will answer the question "How can we turn 3D scans into CAD-quality 3D assets?".