Is There a Universal Dimensionality Reduction Technique for Feature Extraction?–A Comparative Analysis

The demand for high-dimensional data processing in machine learning has led to the increasing use of dimensionality reduction techniques. These techniques aim to extract the most important information from high-dimensional data, reducing it to a lower-dimensional representation that can be easily processed by machine learning algorithms. However, with the availability of a multitude of dimensionality reduction techniques and heterogeneous datasets, it can be challenging for researchers to select the most appropriate one for their specific application. This research conducts a comparative analysis to identify the distinctive behaviors of various dimensionality reduction techniques under different data situations. The state-of-the-art linear and non-linear dimensionality reduction techniques are analyzed. The study also analyses the performance of each technique in terms of its ability to extract meaningful, interpretable, and low-dimensional features from high-dimensional data. The analysis results provide insights into each technique's strengths and weaknesses and highlight the most appropriate technique when dealing with heterogeneous datasets for different machine-learning tasks. We use multiple tabular, text, and image datasets to validate our findings.