Cookie Consent
Hi, this website uses essential cookies to ensure its proper operation and tracking cookies to understand how you interact with it. The latter will be set only after consent.
Read our Privacy Policy

Feature Selection

Feature selection, also known as variable selection or attribute selection, is a process in Machine Learning where you selectively choose those features which contribute most to your prediction variable or output in which you are interested. The main goal of this process is three-fold: improving the prediction performance of the models, providing faster and more cost-effective models, and providing a better understanding of the underlying process that generated the data.

How Feature Selection works

The feature selection process works by selecting a subset of the original features based on certain criteria like the correlation with the target, the increase in model's accuracy, or other statistical properties. Different strategies for feature selection include:

1. Filter Methods‚Äć

These methods use statistical measures to score the relevance of individual features with respect to the target variable, like Chi-Squared Test, information gain, and correlation coefficient scores.

2. Wrapper Methods‚Äć

These methods use a machine learning model to score the utility of a subset of features, which means the search for the best feature subset is "wrapped" around the algorithm, like recursive feature elimination.

3. Embedded Methods‚Äć

These methods perform feature selection as a part of the model construction process, like LASSO and Ridge regression.

Feature selection can give you a boost in accuracy by both enabling the algorithms to train faster (as there are fewer data to process) and reducing overfitting (by removing redundant features).

Lakera LLM Security Playbook
Learn how to protect against the most common LLM vulnerabilities

Download this guide to delve into the most common LLM security risks and ways to mitigate them.

Related terms
untouchable mode.
Get started for free.

Lakera Guard protects your LLM applications from cybersecurity risks with a single line of code. Get started in minutes. Become stronger every day.

Join our Slack Community.

Several people are typing about AI/ML security. 
Come join us and 1000+ others in a chat that’s thoroughly SFW.