In plain words
Parameters are values inside a model that determine how it processes inputs. In neural networks they include weights, biases, and other learned values. Training adjusts them to improve the model’s objective.
A closer look
A parameter count describes how many learned values a model has. More parameters can provide greater capacity, but performance also depends on architecture, training data, computation, and the task. For mixture-of-experts models, the total parameter count can be much larger than the number used for a particular token.
Parameters differ from hyperparameters, which configure learning or architecture: examples include the learning rate, batch size, and number of layers. In everyday product discussions, “parameters” can also mean API settings. It helps to ask whether someone means learned values or options chosen when running the model.
In practice
In the simple prediction y = wx + b, the weight w and bias b are learned parameters. A learning rate chosen to train that model is a hyperparameter.
A useful distinction
A larger parameter count is not a guaranteed ranking of intelligence or usefulness. A smaller model tuned for a specific task may outperform a larger general model on that task while being cheaper to run.