🏗️ 3D Reconstruction from Single Images

⚠️ Responsible Use Guidelines

Privacy & Consent

• Do not upload images containing identifiable people without their explicit consent
• Do not use for surveillance, tracking, or monitoring individuals
• Facial features may be reconstructed in 3D - consider privacy implications
• Remove metadata (EXIF) that may contain location or personal information

Ethical Use

• This tool is for educational, research, and creative purposes only
• Prohibited uses:
  • Creating deepfakes or misleading 3D content
  • Unauthorized documentation of private property
  • Circumventing security systems
  • Generating 3D models for harassment or stalking
  • Commercial use without proper rights to source images

Limitations & Bias

• Models trained primarily on indoor Western architecture
• May perform poorly on non-Western architectural styles
• Scale is relative, not absolute - not suitable for precision measurements
• Single viewpoint limitations - occluded areas are inferred, not captured

Data Usage

• Images are processed locally during your session
• No images are stored or transmitted to external servers
• Processing logs contain only technical metrics, no image content
• You retain all rights to your uploaded images and generated 3D models

By using this tool, you agree to these responsible use guidelines.

Known Limitations & Biases

• Trained primarily on Western indoor architecture
• May underperform on non-Western styles
• Scale is relative, not absolute
• Single viewpoint captures only visible surfaces

About This Tool

This application demonstrates how artificial intelligence can convert single 2D photographs into interactive 3D models automatically.

What Makes This Special

Traditional Approach:

• Need special equipment (3D scanner, multiple cameras)
• Requires technical expertise
• Time-consuming process
• Expensive

This AI Approach:

• Works with any single photograph
• No special equipment needed
• Automatic processing
• Free and accessible

The Technology

AI Model Used: GLPN

GLPN (Global-Local Path Networks)

• Paper: Kim et al., CVPR 2022
• Optimized for: Indoor/outdoor architectural scenes
• Training: NYU Depth V2 (urban indoor environments)
• Best for: Building interiors, street-level views
• Speed: Fast (~0.3-2.5s)

How It Works (Simplified)

1. AI analyzes photo → Recognizes objects, patterns, perspective
2. Estimates distance → Figures out what's close, what's far
3. Creates 3D points → Places colored dots in 3D space
4. Builds surface → Connects dots into smooth shape

Spatial Data Pipeline

1. Monocular Depth Estimation

• Challenge: Extracting 3D spatial information from 2D photographs
• Application: Similar to photogrammetry but from single images
• Output: Relative depth maps for spatial analysis

2. Point Cloud Generation

• Creates 3D coordinate system (X, Y, Z) from pixels
• Each point: Spatial location + RGB color information
• Compatible with: GIS software, CAD tools, spatial databases

3. 3D Mesh Generation

• Creates continuous surface from discrete points
• Similar to: Digital terrain models (DTMs) for buildings
• Output formats: Compatible with ArcGIS, QGIS, SketchUp

Quality Metrics Explained

• Point Cloud Density: Higher points = better spatial resolution
• Geometric Accuracy: Manifold checks ensure valid topology
• Surface Continuity: Watertight meshes = complete volume calculations
• Data Fidelity: Triangle count indicates level of detail

Limitations for Geographic Applications

1. Scale Ambiguity: Requires ground control points for absolute measurements
2. Single Viewpoint: Cannot capture occluded facades or hidden spaces
3. No Georeferencing: Outputs in local coordinates, not global (lat/lon)
4. Weather Dependent: Best results with clear, well-lit conditions

Comparison with Traditional Methods

vs. Terrestrial Laser Scanning (TLS):

• Much cheaper, faster, more accessible
• Lower accuracy, no absolute scale

vs. Photogrammetry (Structure-from-Motion):

• Works with single image, faster processing
• Less accurate, cannot resolve scale

vs. LiDAR:

• Much lower cost, consumer cameras sufficient
• Lower precision, no absolute measurements

Reconstruction Pipeline (10 Steps)

1. Image Preprocessing: Resize to model requirements
2. Depth Estimation: Neural network inference
3. Depth Visualization: Create comparison images
4. Point Cloud Generation: Back-project using camera model
5. Outlier Removal: Statistical filtering
6. Normal Estimation: Surface orientation calculation
7. Mesh Reconstruction: Poisson surface reconstruction
8. Quality Metrics: Compute geometric measures
9. 3D Visualization: Create interactive viewer
10. File Export: Generate multiple formats

Key References

1. Kim, D., et al. (2022). "Global-Local Path Networks for Monocular Depth Estimation with Vertical CutDepth." CVPR 2022
2. Kazhdan, M., et al. (2006). "Poisson Surface Reconstruction." Eurographics Symposium on Geometry Processing

How to Use This Application

Step 1: Read Responsible Use Guidelines

• REQUIRED: Review the "Responsible Use" tab first
• Understand privacy implications
• Acknowledge model limitations and biases
• Ensure you have rights to use source images

Step 2: Prepare Your Image

Best Practices:

• Remove EXIF metadata (GPS, timestamps) for privacy
• Ensure you have consent if image contains people
• Use well-lit, clear photographs
• Recommended resolution: 512-1024 pixels
• Indoor scenes work best

Privacy Checklist:

• No identifiable people (or consent obtained)
• No sensitive/private locations
• EXIF data removed
• You own rights to the image

Step 3: Upload Image

• Click "Upload Image" area
• Select JPG, PNG, or BMP file
• Note: Webcam option removed for privacy protection
• You can also paste from clipboard

Step 4: Check Consent Box

• Check "I have read and agree to Responsible Use Guidelines"
• This confirms you've reviewed ethical guidelines
• Processing won't start without consent

Step 5: Choose Visualization

• Mesh: Solid 3D surface (recommended)
• Point Cloud: Individual 3D points with colors

Step 6: Start Reconstruction

• Click "Start Reconstruction"
• Processing takes 10-60 seconds
• All processing is local (no cloud upload)

Step 7: Explore Results

Depth Map:

• Yellow/Red = Farther objects
• Purple/Blue = Closer objects
• Shows AI's depth understanding

3D Viewer:

• Rotate: Click and drag
• Zoom: Scroll wheel
• Pan: Right-click and drag
• Reset: Double-click

Metrics Report:

• Processing performance
• Quality indicators
• Topology validation

Step 8: Download Files

• ZIP package contains:
  • Point cloud (PLY)
  • Mesh (PLY, OBJ, STL)
  • Quality metrics (JSON)
• All files include responsible AI metadata

Viewing Downloaded 3D Files

Free Software Options:

MeshLab (Recommended for beginners)

• Download: https://www.meshlab.net/
• Open PLY, OBJ, STL files
• Great for viewing and basic editing

Blender (For advanced users)

• Download: https://www.blender.org/
• Import → Wavefront (.obj) or PLY
• Full 3D modeling and rendering capabilities

CloudCompare (For point clouds)

• Download: https://www.cloudcompare.org/
• Best for analyzing point cloud data
• Measurement and analysis tools

Online Viewers (No installation)

• https://3dviewer.net/
• https://www.creators3d.com/online-viewer
• Just drag and drop your OBJ/PLY file

Tips for Best Results

DO:

• Use well-lit images
• Include depth cues (corners, edges)
• Indoor scenes work best
• Medium resolution (512-1024px)
• Remove personal metadata
• Obtain consent for people in images

AVOID:

• Motion blur or low resolution
• Reflective surfaces (mirrors, glass)
• Images without consent
• Private property without permission
• Surveillance or monitoring purposes
• Heavy shadows or darkness

Understanding the Metrics

Point Cloud Statistics:

• Initial Points: Raw points generated from depth
• Outliers Removed: Noisy points filtered out (typically 5-15%)
• Final Points: Clean points used for mesh generation

Mesh Quality Indicators:

• ** Edge Manifold**: Each edge connects exactly 2 faces (good topology)
• ** Vertex Manifold**: Clean vertex connections
• ** Watertight**: No holes, ready for 3D printing
• ** Marks**: Indicate potential issues (still usable, may need repair)

Processing Times:

• Depth Estimation: 0.3-2.5s (GLPN model)
• Mesh Reconstruction: 2-10s (depends on point cloud size)
• Total Time: Usually 10-60 seconds

Troubleshooting

Problem: No output appears

• Check browser console for errors
• Try refreshing the page
• Try a smaller/simpler image first
• Check that image uploaded successfully

Problem: Mesh has holes or artifacts

• This is normal for single-view reconstruction
• Hidden surfaces cannot be reconstructed
• Use mesh repair tools in MeshLab if needed

Problem: Colors look wrong on mesh

• Vertex color interpolation is approximate
• This is expected behavior
• Colors on point cloud are more accurate

Problem: Processing is very slow

• Use smaller images
• This is normal on CPU (GPU is much faster)

Problem: "Not watertight" in metrics

• Common for complex scenes
• Still usable for visualization
• For 3D printing: use mesh repair in MeshLab

Algorithmic Bias & Fairness

Training Data Representation

Geographic Bias:

• Heavy representation: North America, Europe
• Underrepresented: Africa, South Asia, Pacific Islands
• Impact: Lower accuracy for non-Western architecture

Architectural Style Bias:

• Well-represented: Modern interiors, Western buildings
• Underrepresented: Traditional, vernacular, indigenous structures
• Impact: May misinterpret non-standard spatial layouts

Socioeconomic Bias:

• Training data skewed toward middle/upper-class interiors
• Limited representation of informal settlements
• May not generalize well to all socioeconomic contexts

Potential Harms

** Privacy Violations:**

• Unauthorized 3D reconstruction of private spaces
• Creating models of individuals without consent
• Surveillance and tracking applications

** Misinformation:**

• Generating fake 3D evidence
• Manipulating spatial understanding
• Creating misleading visualizations

** Property Rights:**

• Unauthorized documentation of copyrighted designs
• Intellectual property theft
• Commercial exploitation without permission

Harm Prevention

1. Mandatory consent: Require user acknowledgment
2. Use case restriction: Prohibit surveillance and deceptive uses
3. Privacy protection: Disable webcam, encourage EXIF removal
4. Transparency: Clear documentation of limitations

Accountability & Governance

User Responsibilities

As a user, you are responsible for:

• Ensuring lawful use of source images
• Obtaining necessary consents and permissions
• Respecting privacy and intellectual property
• Using outputs ethically and transparently
• Understanding and accounting for model biases

Developer Responsibilities

This tool implements:

• Clear responsible use guidelines
• Privacy-protective design (no webcam, local processing)
• Bias documentation and transparency
• Prohibited use cases explicitly stated

Future Directions

Improving Fairness

• Train on more diverse geographic datasets
• Include underrepresented architectural styles
• Develop bias mitigation techniques
• Community-driven model evaluation

Enhancing Privacy

• Face/person detection and redaction
• Automatic EXIF stripping
• Differential privacy techniques

Academic Citation

For GLPN Model:

@inproceedings{kim2022global,
  title={Global-Local Path Networks for Monocular Depth Estimation with Vertical CutDepth},
  author={Kim, Doyeon and Ga, Woonghyun and Ahn, Pyungwhan and Joo, Donggyu and Chun, Sehwan and Kim, Junmo},
  booktitle={CVPR},
  year={2022}
}

For Poisson Surface Reconstruction:

@inproceedings{kazhdan2006poisson,
  title={Poisson Surface Reconstruction},
  author={Kazhdan, Michael and Bolitho, Matthew and Hoppe, Hugues},
  booktitle={Symposium on Geometry Processing},
  year={2006}
}

Open Source Components

This application is built with:

• Transformers (Hugging Face): Model inference framework
• Open3D: Point cloud and mesh processing
• PyTorch: Deep learning framework
• Plotly: Interactive 3D visualization
• Gradio: Web interface framework
• NumPy & SciPy: Numerical computing
• Matplotlib: Data visualization
• Pillow (PIL): Image processing

Model Credits

GLPN Model:

• Developed by: KAIST (Korea Advanced Institute of Science and Technology)
• Hosted by: Hugging Face (vinvino02/glpn-nyu)
• License: Apache 2.0

Responsible AI Features

This implementation includes:

• Privacy-protective design (no webcam option)
• Mandatory consent acknowledgment
• Bias documentation and transparency
• Ethical use guidelines