SLMs can be moar 💫

For researchers interested in how language models do the things they do, smaller models offer an inexpensive way to test novel ideas. 

And because they have fewer parameters than large models, their reasoning might be more transparent. 

“If you want to make a new model, you need to try things,” said Leshem Choshen, a research scientist at the MIT-IBM Watson AI Lab. “Small models allow researchers to experiment with lower stakes.”

The big, expensive models, with their ever-increasing parameters, will remain useful for applications like generalized chatbots, image generators, and drug discovery. But for many users, a small, targeted model will work just as well, while being easier for researchers to train and build. 

“These efficient models can save money, time and compute,” Choshen said.