2022–23

For 2022-23, the projects listed below are proposed for Year 5.  The 2022-23 project list will be updated once Year 5 projects are selected. 2022-2023

• F2-T DeepSLAM: Object Detection, Re-identification and Prediction with Implicit Mapping
  We first theoretically explore the impact of neural quantization on federated knowledge transfer across quantized DNNs and provide the convergence proof of the quantized federated learning.
  Project Details
• F6-M Adaptive Manifold Learning for Multi-Sensor Translation and Fusion given Missing Data
  This project aims to translate streams of data from individual sensors into a shared manifold-space for joint understanding and processing. This work includes an investigation of computational topology and contrastive learning for manifold learning.
  Project Details
• M1-T Point Cloud Unsampling and Compression (Point Cloud Denoising)
  The goal of this project is to advance the point-cloud post-processing using deep learning method to understand the global and local manifolds of a 3D object.
  Project Details
• M2-V Video De-Duplication
  The goal of this project is to develop novel deep learning algorithms for video segment hashing and identification to support efficient and accurate duplicates identification and removal from phones and cloud storages. 
  Project Details
• M8-M/C Secure Inner Product for privacy preserving pattern matching
  The goal of the project is to secure authentication of a template, especially a biometric query, without compromising the template, the database, or the query; in case of database attack or a corrupted communication channel.
  Project Details
• M10 Grid Congestion Price Forecasting using Deep Learning
  The objective of this project is to provide an accurate electricity day ahead price forecasting system in presence of congestions; using data comprising of power generation from various energy plants, weather conditions, and past nodal prices; by adoption of modern deep learning techniques.
  Project Details
• M11 Machine Learning for Federated Heterogeneous Communication Networks via 5G and 6G  Network Technologies
  The goal of the project is to develop a comprehensive framework for multi-parameter optimization of Heterogeneous Communication Networks for 5G and beyond.
  Project Details
• M12 AI-Driven Precision Music DJing for Elderlies and People in Need
  The objective of this project is for AI to select the best music for elderlies and people in need. The success of this project will lead to some amazingly transformative results in the treatment of dementia through music.
  Project Details
• O1-H Semantic Deep Learning for Electronic Health Records
  We propose a novel “Semantic Deep Learning” method to analyze the electronic health records of real patients. Our previous work as successfully used a hypergraph- based approach in the clinical text notes from Stanford Hospital’s Clinical Data Warehouse (STRIDE). Previous experiments based on ontology (i.e., domain knowledge) annotated electronic health records show that hypergraph mining ...
  Project Details
• O2-M Ontology-based Deep Learning with Explanation for Human Behavior Prediction
  In this project, we will investigate ontology-based deep learning (OBDL) algorithms to predict and explain human behaviors in health domains. The main idea of our algorithms is to consider domain knowledge in the design of deep learning models and utilize domain ontologies for explaining the deep learning models and results.
  Project Details
• O3-N Understand Video Transcripts in Live Streamed Videos
  We propose comprehensive resources and models for understanding automatically transcribed videos. In particular, in this project, we pursue a deep learning model for identifying the important points and questions mentioned in a video transcript. To achieve this objective, we employ two specific deep learning models.
  Project Details

2021–22

• F2-T DeepSLAM: Object Detection, Re-identification and Prediction wih Implicit Mapping
  The project goals and objectives are 1.Communication privacy and security of multi-agent systems: Develop a privacy-enhanced multi-agent system that uses shared knowledge for both (i) Vision and (ii) Communication tasks. 2.Ego-motion prediction under Intermittent Feedback: This goal removes the assumption that the GPS signal is always given and considers a GPS denied area. We design a hybrid system to ...
  Project Details
• F6-M Adaptive Manifold Learning for Multi-Sensor Translation and Fusion given Missing Data
  The goal of this work is to translate streams of data from individual sensors into  a shared-manifold space for joint understanding and processing.
  Project Details
• F12-A Explainable Commonsense Question Answering
  Current question answering systems is incapable of providing human-interpretable explanations or proof to support the decision. In this project, we propose general methods to answer common sense questions, offering natural language explanations or supporting facts. In particular, we propose Copy-explainer that generate natural language explanation that later help answer commonsense questions by leveraging structured and ...
  Project Details
• M1-V Deep Learning in Video Compression
  We are proposing end-to-end video compression with motion field prediction. In video-based point cloud compression (V-PCC), a dynamic point cloud is projected onto geometry and attribute videos patch by patch for compression. We propose a CNN-based occupancy map recovery method to improve the quality of the reconstructed occupancy map video. To the best of our ...
  Project Details
• M2-T Point Cloud Denoising
  The goal of this project is to advance the point-cloud post-processing using deep learning method to understand the global and local manifolds of a 3D object.
  Project Details
• M3-V Video De-Duplication
  The goal of this project is to develop novel deep learning algorithms for video segment hashing and identification to support efficient and accurate duplicates identification and removal from phones and cloud storages. 
  Project Details
• M7 P2PDL: Peer-to-Peer Deep Learning using Blockchain for Effective Domain Adaptation & Privacy Preserving
  This project explores domain adaptation & online learning for model customization considering privacy preservation and secure communication. We propose blockchain-based peer-to-peer federated learning (P2PDL) using federated learning applications.
  Project Details
• M8-M/C Secure Inner Product for privacy preserving pattern matching
  The goal of the project is to secure authentication of a template, especially a biometric query, without compromising the template, the database, or the query; in case of database attack or a corrupted communication channel.
  Project Details
• M10 Grid Congestion Price Forecasting using Deep Learning
  The objective of this project is to provide an accurate electricity day ahead price forecasting system in presence of congestions; using data comprising of power generation from various energy plants, weather conditions, and past nodal prices; by adoption of modern deep learning techniques.
  Project Details
• O1-H Semantic Deep Learning for Electronic Health Records
  We propose a novel “Semantic Deep Learning” method to analyze the electronic health records of real patients. Our previous work as successfully used a hypergraph- based approach in the clinical text notes from Stanford Hospital’s Clinical Data Warehouse (STRIDE). Previous experiments based on ontology (i.e., domain knowledge) annotated electronic health records show that hypergraph mining ...
  Project Details
• O2-M Ontology-based Deep Learning with Explanation for Human Behavior Prediction
  In this project, we will investigate ontology-based deep learning (OBDL) algorithms to predict and explain human behaviors in health domains. The main idea of our algorithms is to consider domain knowledge in the design of deep learning models and utilize domain ontologies for explaining the deep learning models and results.
  Project Details
• O3-N Understand Transcripts with Natural Language Processing and Deep Learning
  We propose comprehensive resources and models for understanding automatically transcribed videos. In particular, in this project, we pursue a deep learning model for identifying the important points and questions mentioned in a video transcript. To achieve this objective, we employ two specific deep learning models.
  Project Details

2020–21

• F2-T DeepSLAM: Object Detection, Re-identification and Prediction wih Implicit Mapping
  DeepSLAM project aims to Develop an end-to-end deep neural network simultaneously capable of (a) Object Detection, and (b) Re-identification/Tracking (a) Object Detection using RGB images/videos. (b) Object Re-identification over multiple frames. This all-in-one solution provides a better level of information integrity and reuse. In doing so, a local belief of the surrounding area is trained with occupancy grid ...
  Project Details
• F6-M Adaptive Manifold Learning for Multi-Sensor Translation and Fusion given Missing Data
  The goal of this work is to translate streams of data from individual sensors into a shared manifold-space for joint understanding and processing. This work includes investigation of computational topology for manifold learning, data summarization, and intrinsic dimensionality estimation. In practice, for a given application, processing chains are generally developed for a particular sensor or ...
  Project Details
• F9-V Quasi-Supervised-Learning for SAR Image Recognition
  Current synthetic aperture radar (SAR) image recognition systems suffer from significant degradation when the systems are trained with synthetic images but tested with real measured images. To address this issue, the project is aimed at developing a quasi-supervised-learning approach for SAR image recognition. The key idea is transfer learning with quasi-supervised training procedures.
  Project Details
• F10-V Robust PCA with Outlier Mitigation
  Principal Component Analysis (PCA) has been widely used in computer vision and machine learning applications due to its excellent performance in compression, feature extraction and feature representation. However, PCA suffers from severely degraded performance when outliers exist in datasets. To address this issue, the project is intended to develop a robust PCA algorithm, capable ...
  Project Details
• M1-V DeepCompression: Deep Learning in Future Video Coding
  Conventional video coding methods optimize each part separately which might lead to sub-optimal solution. Motivated by the success of deep learning on computer vision tasks, we are proposing deep learning for video compression in an end-to-end manner.
  Project Details
• M2-T Point Cloud Denoising
  We propose to tackle denoising problem using deep learning into two folds: outlier removal and denoising. We used two stage deep learning pipeline where first stage acts as a binary classifier that classifies 3d points either as outlier or non-outlier. The second stage receives non-outlier and noisy points from the first stage and learns the ...
  Project Details
• M6-A Deep Learning for Energy Usage Prediction in Sustainable Building Design
  The objectives of this research are: (1) Develop a unified and accurate deep learning based regression model for predicting energy usage of different building types. (2) Develop a zero shot learning approach (using an ontology of building types) to accurately predict energy usage of building types whose data are not available during training. (3) Evaluate ...
  Project Details
• O1-H Semantic Deep Learning for Electronic Health Records
  We propose a novel “Semantic Deep Learning” method to analyze the electronic health records of real patients. Our previous work as successfully used a hypergraph- based approach in the clinical text notes from Stanford Hospital’s Clinical Data Warehouse (STRIDE).
  Project Details
• O2-M Ontology-based Deep Learning with Explanation for Human Behavior Prediction
  The goal of this project is to evaluate contemporary techniques for deep learning model explanations and utilize DL Explanation approach for improving model interpretability.
  Project Details
• O3-B Ontology-based Interpretable Deep Learning for Consumer Complaint Explanation and Analysis
  In this project, we will investigate ontology-based explainable deep learning (OBDL) algorithms for textual data to identify key factors, concepts, and hypothesis that significantly contribute to the decision made by deep neural networks.
  Project Details
• O6-N Knowledge Graph based Question Answering for Automatic Customer Service
  Short text Matching plays an important role in natural language processing tasks such as information retrieval (IR), question answering (QA), and dialogue system. Traditional text matching methods rely on human-crafted rules and template. Though they are effective to deal with specific situations, but lack in the ability to handle unobserved cases. In natural language, it’s ...
  Project Details

2019–20

• F2-T DeepSLAM: Object Detection, Re-identification and Prediction wih Implicit Mapping
  DeepSLAM project is designed to develop an end-to-end and all-in-one deep neural network capable of followings, simultaneously, with only road scene RGB images (a) Object Detection using RGB images and videos; (b) Object Re-identification over multiple frames, with occlusions; (c) Predicting multiple object’s future trajectories. This all-in-one solution provides a better level of information integrity and reuse. In doing ...
  Project Details
• F3-N OneTask: Online Intelligent Platform for Education and Business
  The goal of this project is to design and develop an intelligent task management for online education and business purpose. The project includes OneTask online platform, self-adaptive neural question generation for study quality evaluation and companion, GCN-based DRL model for self-adaptive neural question generation and study content recommendation and entity-related open-domain question answering system.
  Project Details
• F4-B DeepFin: Federated Learning for Risk Management with Transparency and Accountability
  The goal of this project is to develop a Deep Neural Network based risk management model that can help financial companies predict loan default likelihood with a higher accuracy when a customer applies for a loan.
  Project Details
• F5-C FedSec: Federated Learning Security Attacks and Defenses
  The goal of this project is to explore potential vulnerabilities in federated learning applications. Federated learning is a new kind of distributed machine learning with decentralized data. There is no need for data sharing for federated learning.
  Project Details
• F6-M Adaptive Manifold Learning for Multi-Sensor Translation and Fusion given Missing Data
  The goal of this work is to translate streams of data from individual sensors into a shared manifold-space for joint understanding and processing. This work includes investigation of computational topology for manifold learning, data summarization, and intrinsic dimensionality estimation. In practice, for a given application, processing chains are generally developed for a particular sensor or ...
  Project Details
• F8-M TPR Learning: a Symbolic Neural Approach for Vision Language Intelligence
  Deep learning (DL) has in recent years been widely used in computer vision and natural language processing (NLP) applications due to its superior performance. However, while images and natural languages are known to be rich in structures expressed, for example, by grammar rules, DL has so far not been capable of explicitly representing and enforcing ...
  Project Details
• M1-V Deep Learning in Video Compression
  Advancing the state-of-the-art in image/video compression by adopting deep learning methods in prediction, transform, entropy coding and post processing. Develop fresh new coding tools based on deep learning for post processing, reconstruction enhancement. Investigate new pipelines using deep learning for end-to-end image/video compression. Achieve significant coding improvements with applicable computational complexity as well as deliver ...
  Project Details
• M2-T Deep Learning for Point Cloud Denoising (renamed: 3D Point Cloud Denoising and Compression using Deep Learning)
  Enable efficient 3D sensing and information sharing for auto-driving and smart city with 3D point cloud denoising and end-to-end compression using deep learning architectures. We would explore deep learning architecture for point cloud processing. We formulate a multi-scale CNN based 3d point cloud feature extraction technique.
  Project Details
• M3-V Privacy-Preserving Fall Detection with Deep Learning on mmWaveRadar Signal (renamed: Low-Light and Low-Resolution Image Enhancement)
  This project aims to design a deep learning based end-to-end solution for dark image denoising and enhancement using image decomposition method, achieving significant improvement in output image quality. Further it aims to improve the image super-resolution by exploiting temporal redundancy. The project also aims at developing an end-to-end solution for multi-frame image super-resolution which involves ...
  Project Details
• M5-S ModelKB: Automated Deep Learning Management System
  We propose the ModelKB system automating end-to-end model management in deep learning. We will develop a ModelKB prototype that can automatically (1) extract and store the model’s metadata-including its architecture, weights, and configuration; (2) visualize, query, and compare experiments; and (3) reproduce experiments.
  Project Details
• O1-N Semantic Deep Learning for Electronic Health Records
  We propose a novel “Semantic Deep Learning” method to analyze the electronic health records of real patients. Our previous work as successfully used a hypergraph- based approach in the clinical text notes from Stanford Hospital’s Clinical Data Warehouse (STRIDE).
  Project Details
• O2-M Ontology-based Deep Learning with Explanation for Human Behavior Prediction
  In the second year, we will focus on domain ontology-specific deep learning methods, integrating contemporary deep learning model designs with the existing SMASH healthcare ontology and corresponding data, to structure both model input and potentially output in order to develop meaningful semantic explanations that compare favorably with the state of the art.
  Project Details
• O3-B Ontology-based Interpretable Deep Learning for Consumer Complaint Explanation and Analysis
  In this project, we will design ontology-based interpretable deep models for consumer complaint explanation and analysis. The main idea of our algorithms is to consider domain knowledge in the design of deep learning models and utilize domain ontologies for explaining the deep learning models and results through casual modeling.
  Project Details
• O4-N Multilingual Knowledge Alignment with Embedding Representation Learning
  In this project, we will develop innovative medical embedding learning algorithms   and make knowledge alignment across multiple languages in the medical domain. Our learning algorithms will fully exploit the semantic similarity for knowledge alignment across languages.
  Project Details
• O6-N Knowledge Graph based Question Answering for Automatic Customer Service
  The objectives of this project involve developing a customer service ontology represented as knowledge graph and building an automatic customer service system backed by the knowledge graph.
  Project Details

2018–19

• B1-S DeepCloud: An Intelligent Platform by the Community and for the Community
  DeepCloud is designed as an open software-defined ecosystem for researchers at different levels with the following salient features and transformative impacts. It is one of the first massively scalable multi-tenant open cloud platform with full-fledged building blocks and comprehensive shared stores (app, model, knowledge, data) for deep learning research and applications.
  Project Details
• F1-M Bidirectional Deep Learning Architecture for Scene Understanding
  This project aims at creating deep architectures inspired by cognitive sciences to under visual scenes either in images or videos. The characteristic of the proposed architecture is that it simplifies the inference using biological plausible marginals (object type and spatial location), which can be learned in an unsupervised way directly from data (i.e. without labels).
  Project Details
• F2-M GraphBTM: Graph Enhanced VAE for Biterm Topic Model
  GraphBTM is a topic model which is an unsupervised algorithm to understand documents. It learns to discover the latent representation of documents and produce meaning clustering of words in the same topic. The goal of GraphBTM is to overcome the limitations of the Latent Dirichlet Allocation (LDA) which suffers from the data sparsity problem in ...
  Project Details
• F3-M TPR Learning: a Symbolic Neural Approach for Vision Language Intelligence
  Deep learning (DL) has in recent years been widely used in computer vision and natural language processing (NLP) applications due to its superior performance. However, while images and natural languages are known to be rich in structures expressed, for example, by grammar rules, DL has so far not been capable of explicitly representing and enforcing ...
  Project Details
• F4-M Adaptive Manifold Learning for Multi-Sensor Translation and Fusion given Missing Data
  The goal of this work is to translate streams of data from individual sensors into a shared manifold-space for joint understanding and processing. This work includes investigation of computational topology for manifold learning, data summarization, and intrinsic dimensionality estimation. In practice, for a given application, processing chains are generally developed for a particular sensor or ...
  Project Details
• F5-T DeepSLAM: Visual Intelligence for Navigation and Planning
  The goals of this project include developing the ability to use offline, transferred, and real-time learning using various data sources (intermittent state feedback, cloud) to enable SLAM and related image-based estimation methods.
  Project Details
• F6-V Machine-Learning-Enabled Video Coding Strategy for Object Detection
  The goal of this project is to develop a machine-learning-enabled video coding strategy for object detection.  Most existing video encoders minimizes distortion under a rate constraint. However, for surveillance video, it is desired for a video encoder to maximize detection probability under a rate constraint. To address this, we will design a new video coding ...
  Project Details
• F7-S One Task: An Intelligent Assistant Platform
  The goals of this project are to design and develop an Intelligent Task Management Platform with task-oriented assistants for commercial and education services, design and develop an open-domain question answering system, and design and develop a deep neural model for question answering with external Knowledge Base.
  Project Details
• M1-V Deep Learning for Future Video Compression
  The goal of this project is to develop new Deep Learning based high dimensional signal models and prediction tools for immersive visual signal coding.
  Project Details
• M2-S Smart-ARM: Smart Streaming Telemetry for Agile and Resilient Management of Wireless and Mobile Software Defined Networking
  Traditional application-driven (i.e., fault, SLA, or DoS attack detection) “pull” model (SNMP) based network management is expensive and slow for network problem detection, isolation, and root cause analysis. Especially, the recent federation of novel softwareisation and virtualization architectures, as well as Internet of Things (IoT) technologies, require better management over the heterogeneous systems and services. ...
  Project Details
• M3-T Mobile Edge Point Cloud Services for Auto Driving
  Provide Realtime live 3D map service from mobile edge using distributed sensing and low latency point cloud aggregation and multicasting. We will use high efficiency scalable point cloud source coding. We will provide real time point cloud LiveMaps aggregated through mobile edge computing. We propose a joint source-channel coding for V2V and V2I communication.
  Project Details
• M4-V Querying Images via Knowledge Graphs
  The objective of this project is to develop QIK, a fast graph-based query processing system for searching images via metadata using the concept of knowledge graphs. QIK will also recommend interesting questions to users to enable easier search.
  Project Details
• M5-M Low Resolution and Quality Image Understanding
  The goals of this project are (1) to make new end to end image processing pipeline that performs the extremely low light image denoising and enhancement task and (2) to develop a recognition friendly super resolution method for low resolution image recognition.
  Project Details
• O1-N Semantic Deep Learning for Electronic Health Records
  The goal of this project is to create a novel “Semantic Deep Mining” method to analyze the electronic health records (EHRs) of real patients.
  Project Details
• O2-M Ontology-based Deep Learning with Explanation for Human Behavior Prediction
  The goals of this project are to (1) evaluate contemporary techniques for deep learning model explanations and (2) utilize DL Explanation approach for improving model performance.
  Project Details
• O3-B Ontology-based Interpretable Deep Learning for Consumer Complaint Explanation and Analysis
  The goal of this project is to design ontology-based interpretable deep models for consumer complaint explanation and analysis.
  Project Details
• O4-N Multilingual Knowledge Alignment with Embedding Representation Learning
  The goal of this project is to create a novel “Multilingual Knowledge Alignment” method in the medical domain with no/less parallel corpus, to enhance/improve Medical Knowledge in Chinese.
  Project Details
• O5-B Deep Learning for Improving Stock Market Investment
  The objective of this project is to create data-driven deep learning models that (1) optimizes the institutional investor’s investment strategy in the stock market (2) provides decision support for stock pick 93) are “accountable” and (4) are flexible enough to properly accommodate heterogeneous data.
  Project Details
• O6-T Advanced Product Differentiation using Deep Learning
  The goal of this projects is to prototype to segment 3D objects into equivalence classes using known Deep Learning techniques; understand performance of known techniques.
  Project Details
• O7-H Using Machine Learning to Discover How the Brain Works
  The goals of this projects are to (1) develop ML/DL tools to understand high-resolution temporal/spatial neurological data and (2) use ML/DL to create/refine brain models (mouse, human) to emulate brain dynamics.
  Project Details