Jiayao Zhang 张嘉瑶
Home Projects Publication

Publication & Research

Publication

Depth Estimation for Indoor Panoramas

Depth Estimation for Indoor Panoramas through Merging Multiple Perspective Monocular Depth Predictions

WACV   Nov 2022

We propose a method to estimate depths for indoor panoramas by merging multiple perspective depth maps produced by modern monocular depth estimation methods such as LeReS. The challenge is that the perspective depth maps, each covering a different subset of the panorama, tend to have different scales and shifts of the predicted depth values. Simply stitching them together led to inconsistent depth values for the whole panorama with visible seams. To address the challenge, we propose a novel approach to solve a single depth map for the whole panorama with information taken from each perspective depth estimations and a common panoramic depth map served as the reference for merging.

Distortion Reduction for Off-Center Perspective Projection of Panoramas

Distortion Reduction for Off-Center Perspective Projection of Panoramas

Best Short Paper Award   NICOGRAPH   Nov 2021

In this paper, we propose modifications to the equirectangular-to-perspective (E2P) projection that significantly reduce distortions when the camera position is away from the origin. This enables users to not only "look around" but also "walk around" virtually in a single panorama with more convincing renderings. We compare with other techniques that aim to augment panoramas with 3D information, including 1) panoramas with depth information and 2) panoramas augmented with room layouts, and show that our approach provides more visually convincing results.

More on [Google Scholar]

Research

AI Infra

AI Infra · Chapter 06.03: New Architectures

2023 Open source technical writing · Infrasys-AI · Apache-2.0

Section coverage:

- RNN → Transformer → SSM architectural evolution and efficiency/expressiveness trade-offs

- Structured State Space Models (S4): mathematical foundations, HiPPO matrix, and diagonal-plus-low-rank parameterization

- Mamba selective SSM: input-dependent parameters, hardware-aware parallel scan implementation, and comparison with Transformer/RNN

- MoE-Mamba: mixture-of-experts integration with SSMs; training/inference efficiency gains

NaniteTree Pipeline

NaniteTree Pipeline · Project Page

Invited Talk · Epic Games Shanghai · 2023

An open-source automated pipeline that converts traditional non-Nanite vegetation models into Nanite-compatible geometry for Unreal Engine 5, bridging Houdini Digital Assets and a companion Unreal plugin for end-to-end asset-authoring workflows.

UnrealEngineGPT

UnrealEngineGPT · Project Page

Honor Certificate · InternLM · Shanghai AI Lab · 2024

An end-to-end open-source system combining a domain-specialized LLM for Unreal Engine (documentation Q&A, code generation) with an in-editor agentic plugin that drives Stable Diffusion + ControlNet workflows to generate 3D asset textures directly on selected actors.