Researchers Transfer Stellar Data Using Pre-Trained Models

Scientists are tackling the significant challenge of cross-survey generalisation in stellar spectral analysis, specifically the transfer of knowledge from low- to moderate-resolution spectroscopic surveys. Xiaosheng Zhao, Yuan-Sen Ting, and Rosemary F.G. Wyse, working with colleagues from the Department of Physics & Astronomy at The Johns Hopkins University, the Department of Astronomy at The Ohio State University, the School of Astronomy and Space Science at the University of Chinese Academy of Sciences, and the Department of Applied Mathematics & Statistics at Johns Hopkins University, demonstrate a novel approach using pre-trained neural networks. Their research focuses on leveraging data from the LAMOST low-resolution survey to improve stellar parameter estimation from the DESI medium-resolution survey, a task crucial for large-scale galactic archaeology and understanding stellar populations. By comparing multilayer perceptrons trained directly on spectra with those utilising embeddings from transformer-based models, and evaluating various fine-tuning strategies, the team reveals that simple pre-trained models can achieve competitive results, paving the way for more efficient and accurate stellar analyses across diverse datasets.

Scientists are tackling a longstanding problem in astronomy: reliably classifying stars using differing observational data. Obtaining consistent stellar properties across surveys is vital for building a comprehensive picture of the galaxy. This work demonstrates a surprisingly effective technique using artificial intelligence to bridge the gap between datasets of varying quality.

Scientists are tackling a fundamental challenge in modern astronomy: ensuring consistency when analysing stellar spectra obtained from different surveys. Cross-survey generalisation, the ability to accurately estimate stellar properties like temperature, chemical composition, and surface gravity from data collected with varying instruments and methods, is crucial for maximising the scientific return from the wealth of spectroscopic data now available.

This work presents a detailed investigation into techniques for transferring knowledge gained from the LAMOST low-resolution spectra (LRS) to the more detailed medium-resolution spectra (MRS) provided by the DESI instrument. Researchers have demonstrated that pre-trained multilayer perceptrons (MLPs), a type of neural network, can achieve surprisingly strong performance in this cross-survey setting, even without any adjustments to the model.

Specifically, the study focuses on leveraging models initially trained on the large LAMOST dataset and adapting them for use with DESI spectra. The team compared the effectiveness of training MLPs directly on the raw spectral data versus utilising spectral embeddings, compressed representations of the spectra generated by more complex, self-supervised transformer-based models, often referred to as foundation models.

These foundation models are pre-trained on vast amounts of data to learn general spectral features. The research also explored various fine-tuning strategies, including methods that modify only a small portion of the pre-trained model’s parameters, to optimise performance on the DESI data. Results indicate that while transformer-based embeddings offer advantages when analysing stars with relatively high metal content ([Fe/H] > -1.0), the simpler MLPs trained directly on LAMOST spectra outperform them for metal-poor stars.

This finding suggests that complex foundation models are not always necessary for effective cross-survey generalisation. Furthermore, the optimal approach to fine-tuning the models depends on the specific stellar parameter being estimated. The work highlights the potential of simple, pre-trained neural networks to provide competitive results, while also suggesting that further research is needed to fully understand the role of spectral foundation models in this context. Ultimately, this research paves the way for more efficient and reliable analysis of large-scale spectroscopic datasets, enabling astronomers to build a more complete picture of the Milky Way and the stars within it.

Cross-survey stellar abundance determination using pre-trained multilayer perceptrons and transformer embeddings

Pre-trained multilayer perceptrons demonstrate strong performance in transferring stellar spectral analysis from the LAMOST low-resolution survey to DESI medium-resolution spectra, even without fine-tuning, establishing a baseline for cross-survey generalisation. Modest fine-tuning with DESI spectra subsequently improves these results, indicating the adaptability of the pre-trained models to the new dataset.

Specifically, for iron abundance ([Fe/H]), embeddings derived from a transformer-based model offer advantages only in the metal-rich regime, exceeding the performance of MLPs trained directly on LAMOST spectra when [Fe/H] is greater than -1.0. However, in the metal-poor regime, MLPs trained directly on the low-resolution LAMOST spectra consistently outperform those utilising embeddings from the transformer-based model.

This suggests that the benefits of spectral foundation models are not uniform across the range of stellar metallicities. Further analysis reveals that the optimal fine-tuning strategy is parameter-dependent; different stellar properties benefit from distinct adaptation approaches. Residual-head adapters, LoRA, and full fine-tuning were all evaluated, with the most effective method varying based on the specific stellar parameter being estimated.

These findings highlight the potential of simple pre-trained MLPs as a viable solution for cross-survey generalisation in stellar spectroscopy. These models can achieve competitive results without relying on complex foundation models, at least for certain stellar populations and parameter estimations. While transformer-based models show promise in metal-rich stars, their effectiveness requires further investigation, particularly in the metal-poor regime where simpler architectures currently excel.

Transferring stellar parameter estimates from LAMOST low-resolution to DESI medium-resolution spectra using multilayer perceptrons

Multilayer perceptrons (MLPs) form the core of our investigation into cross-survey generalisation in stellar spectral analysis, specifically addressing the transfer of information from low-resolution to medium-resolution data. We pre-train these MLPs using data from the Large Sky Area Multi-Object Fibre Spectroscopic Telescope (LAMOST) low-resolution spectra (LRS) and subsequently fine-tune them for application to spectra obtained by the Dark Energy Spectroscopic Instrument (DESI) medium-resolution spectra (MRS).

This approach allows us to evaluate the potential of pre-trained models to overcome discrepancies between datasets acquired with differing instrumentation. The MLPs are designed to map spectral information directly to stellar parameters, accepting either continuum-normalized spectra or embeddings derived from transformer-based models as input. When operating on LRS fluxes, the network processes data restricted to 1,462 logarithmically spaced wavelength bins spanning 400-560nm.

The architecture consists of fully connected layers incorporating ReLU activations, culminating in an output layer predicting the target parameter, with approximately 2.06 million trainable parameters. We leverage pre-trained MLPs, originally developed by Zhao et al, to predict the logarithmic abundance ratios of [Fe/H] and [α/Fe]. For the [Fe/H] model, training data comprises 90,106 LAMOST LRS spectra with labels sourced from the APOGEE DR17 catalogue for stars exceeding a metallicity of [Fe/H] > -2.0, supplemented by data from the PASTEL, SAGA, and other VMP/UMP datasets at lower metallicities.

The [α/Fe] model utilizes 85,400 spectra also labelled with APOGEE DR17 data. Data is partitioned into 9:1 training and validation sets, and models are optimised using the AdamW optimizer with a learning rate of 1×10−5, a weight decay of 1×10−4, and a batch size of 32, trained for 100 epochs with validation performance guiding checkpoint selection. To further explore feature representation, we also evaluate the use of spectral embeddings generated by the SpecCLIP framework, a transformer-based model designed to align spectra from diverse surveys by creating a shared representation space. This allows us to compare the performance of MLPs trained directly on spectra with those trained on these foundation-model derived embeddings, assessing the benefits of leveraging pre-trained, self-supervised models.

Machine learning models effectively combine data from differing stellar spectroscopic surveys

Scientists are increasingly reliant on large spectroscopic surveys to map the Milky Way and understand the formation history of our galaxy. However, these surveys aren’t created equal. Differences in instrument resolution and spectral coverage present a significant hurdle when trying to combine data, a process known as cross-survey generalisation. For years, astronomers have struggled to reliably transfer calibrations and analyses between surveys, often requiring painstaking, manual adjustments.

This work offers a promising step towards automating that process, demonstrating that relatively simple machine learning models can bridge the gap between lower and medium-resolution stellar spectra with surprising effectiveness. The success of multilayer perceptrons, even without extensive fine-tuning, is particularly noteworthy. It suggests that a substantial amount of information is preserved in the broad spectral features captured by lower-resolution instruments, allowing models to ‘learn’ and adapt to higher-resolution data.

While more complex transformer-based models show some advantage in specific scenarios, their performance isn’t universally superior, highlighting the need for careful consideration of model choice. The finding that optimal fine-tuning strategies vary depending on the stellar parameter being estimated is also crucial, underlining the lack of a one-size-fits-all solution.

Looking ahead, the focus will likely shift towards refining these machine learning pipelines and exploring the potential of even larger, more diverse datasets. The limitations of current foundation models for metal-poor stars, for example, require further investigation. Ultimately, the goal isn’t just to accurately estimate stellar parameters, but to unlock the full potential of multi-survey datasets, enabling a more complete and nuanced understanding of the galaxy we inhabit.