Marshal Sinaga

selected publications

1. Variational Contrastive Log Ratio Upper Bound of Mutual Information for Training Generative Models

   Sinaga, Marshal, Alhamidi, Machmud Roby, Rachmadi, Muhammad Febrian, and Jatmiko, Wisnu

   2021 6th International Workshop on Big Data and Information Security (IWBIS) 2021

   Abs Bib HTML

   Theoretically, a Generative adversarial network minimizes the Jensen-Shannon divergence between real data distribution and generated data distribution. This divergence is another form of mutual information between a mixture distribution and a binary distribution. It implies that we can build a similar generative model by optimizing the mutual information. This research proposes variational contrastive log-ratio upper bound vCLUB mutual information estimation on mixture distribution and the optimization algorithm to train two generative models. We call the models CLUB-sampling generative network (vCLUBsampling GN) and vCLUB-non sampling generative network (vCLUB-non sampling GN). The results show that vCLUBsampling outperforms GAN and vCLUB-non sampling GN on the MNIST dataset and has competitive results with GAN on the CIFAR-10 dataset. However, GAN outperforms vCLUB-non sampling GN on both datasets.

   @article{sinaga2021club,
     bibtex_show = {true},
     title = {Variational Contrastive Log Ratio Upper Bound of Mutual Information for Training Generative Models},
     author = {Sinaga, Marshal and Alhamidi, Machmud Roby and Rachmadi, Muhammad Febrian and Jatmiko, Wisnu},
     journal = {2021 6th International Workshop on Big Data and Information Security (IWBIS)},
     pages = {9--16},
     year = {2021},
     publisher = {IEEE,},
     html = {https://www.researchgate.net/publication/349881819_Variational_Contrastive_Log_Ratio_Upper_Bound_of_Mutual_Information_for_Training_Generative_Model},
     selected = {true}
   }

2. Tile2Vec with Predicting Noise for Land Cover Classification

   Sinaga, Marshal, Ali, Fadel Muhammad, and Arymurthy, Aniati Murni

   International Conference on Neural Information Processing 2021

   Abs Bib HTML

   Tile2vec has proven to be a good representation learning model in the remote sensing field. The success of the model depends on l2-norm regularization. However, l2-norm regularization has the main drawback that affects the regularization. We propose to replace the l2-norm with regularization with predicting noise framework. We then develop an algorithm to integrate the framework. We evaluate the model by using it as a feature extractor on the land cover classification task. The result shows that our proposed model outperforms all the baseline models.

   @article{sinaga2021tile2vec,
     bibtex_show = {true},
     title = {Tile2Vec with Predicting Noise for Land Cover Classification},
     author = {Sinaga, Marshal and Ali, Fadel Muhammad and Arymurthy, Aniati Murni},
     journal = {International Conference on Neural Information Processing},
     pages = {87--99},
     year = {2021},
     publisher = {Springer,},
     html = {https://www.researchgate.net/publication/356458961_Tile2Vec_with_Predicting_Noise_for_Land_Cover_Classification},
     selected = {true}
   }

3. Transformation Equivariant Representation Learning with Barber Agakov and Info-NCE Mutual Information Estimation

   Sinaga, Marshal, Basarrudin, T., and Krisnadhi, Adila Alfa

   International Conference on Pattern Recognition Applications and Methods (ICPRAM) 2022

   Abs Bib HTML

   The success of deep learning on computer vision tasks is due to the convolution layer that equivaries to the translation transformation. Several works attempt to extend the notion of equivariance into more general transformations. Autoencoding variational transformation (AVT) achieves state of art by approaching the problem from the information theory perspective. The model involves the computation of mutual information, which leads to a more general transformation equivariant representation model. In this research, we investigate the alternatives of AVT called variational transformation equivariant (VTE). We utilize the Barber-Agakov and Info-NCE mutual information estimation to optimize VTE. Furthermore, we also propose a sequential mechanism to train our VTE. Results of experiments demonstrate that VTE outperforms AVT on image classification tasks.

   @article{sinaga2022vte,
     bibtex_show = {true},
     title = {Transformation Equivariant Representation Learning with Barber Agakov and Info-NCE Mutual Information Estimation},
     author = {Sinaga, Marshal and Basarrudin, T. and Krisnadhi, Adila Alfa},
     journal = {International Conference on Pattern Recognition Applications and Methods (ICPRAM)},
     year = {2022},
     publisher = {Scitepress,},
     html = {https://www.researchgate.net/publication/356459026_Transformation_Equivariant_Representation_Learning_with_Barber_Agakov_and_Info-NCE_Mutual_Information_Estimation},
     selected = {true}
   }

4. Preferential Heteroscedastic Bayesian Optimization with Informative Noise Priors

   Sinaga, Marshal, Martinelli, Julien, and Kaski, Samuel

   NeurIPS 2023 Workshop on Adaptive Experimental Design and Active Learning in the Real World 2023

   Abs PDF

   Preferential Bayesian optimization (PBO) is a sample-efficient framework for optimizing a black-box function by utilizing human preferences between two candidate solutions as a proxy. Conventional PBO relies on homoscedastic noise to model human preference struc- ture. However, such noise fails to accurately capture the varying levels of human aleatoric uncertainty among different pairs of candidates. For instance, a chemist with solid expertise in glucose-related molecules may easily compare two compounds and struggle for alcohol- related molecules. Furthermore, PBO ignores this uncertainty when searching for a new candidate, consequently underestimating the risk associated with human uncertainty. To address this, we propose heteroscedastic noise models to learn human preference structure. Moreover, we integrate the preference structure with the acquisition functions that account for aleatoric uncertainty. The noise models assign noise based on the distance of a specific input to a predefined set of reliable inputs known as anchors. We empirically evaluate the proposed approach on a range of synthetic black-box functions, demonstrating a consistent improvement over homoscedastic PBO.