paper            arxiv            github

Score-based generative models (SGMs) is a recent class of deep generative models with state-of-the-art performance in many applications. In this paper, we establish convergence guarantees for a general class of SGMs in 2-Wasserstein distance, assuming accurate score estimates and smooth log-concave data distribution. We specialize our result to several concrete SGMs with specific choices of forward processes modelled by stochastic differential equations, and obtain an upper bound on the iteration complexity for each model, which demonstrates the impacts of different choices of the forward processes. We also provide a lower bound when the data distribution is Gaussian. Numerically, we experiment SGMs with different forward processes, some of which are newly proposed in this paper, for unconditional image generation on CIFAR-10. We find that the experimental results are in good agreement with our theoretical predictions on the iteration complexity, and the models with our newly proposed forward processes can outperform existing models.

arxiv            github

Developed new Gradient Descent Langevin Dynamics by modifying the loss function using smoothing methods and scaling the gradients, specifically targeting non-differentiable or non-convex functions in machine learning applications such as ReLU activation, Lasso, SCAD and MCP regularizers. Fully designed experiments to verify the behaviors of the above algorithms using distributions sampling, Logistic Regression and Neural Networks with Anchored Langevin Gradient Descent algorithms as the optimizer. Obtained convergence with up to 90% average accuracy for practical classification problem and non-convex sampling problems where the existing Gradient Descent Langevin Dynamics fails to converge.

arxiv

Developed self-supervised encoder pretraining framework for audio, image, and video classification tasks, incorporating residual quantization into bidirectional training process, enhancing representation quality. Achieved state-of-the-art results on audio classification benchmarks, and demonstrated competitive performance on image and video datasets.

Doctoral dissertation covering Langevin Dynamics and Diffusion Models for sampling in Machine Learning and Generative AI.

egrove

This thesis is a discussion on the mean-variance approach to portfolio optimization and an introduction of the Bayesian approach, which is designed to solve certain limitations of the classical mean-variance analysis. The primary goal of portfolio optimization is to achieve the maximum return from investment given a certain level of risk. The mean-variance approach, introduced by Harry Markowitz, sought to solve this optimization problem by analyzing the means and variances of a certain collection of stocks. However, due to its simplicity, the mean-variance approach is subject to various limitations. In this paper, we seek to solve some of these limitations by applying the Bayesian method, which is mainly based on probability theory and the Bayes’ theorem. These approaches will be applied to form optimal portfolios using the data of 27 Dow Jones companies in the period of 2008-2017 for a better comparison. The topic of portfolio optimization is extremely broad, and there are many approaches that have been and are being currently researched. Yet, there is no approach that is proven to perform most efficiently. The purpose of this paper is to discuss two potential and popular approaches in forming optimal portfolios.