Feature Engineering for Machine Learning: the best book review

Feature Engineering for Machine Learning: Principles and Techniques for Data Scientists

feature engineering for machine learning pdf

Is Feature Engineering for Machine Learning right for you?

This is a review of Feature Engineering for Machine Learning: Principles and Techniques for Data Scientists by Alice Zheng and Amanda Casari. Its focus is how to represent raw data for a model, including numerical transformations, text features, categorical encoding, and dimensionality reduction.

Best fit: readers who know basic Python and have trained a simple model, but want to understand why input representation matters. Work through one chapter on a small dataset rather than collecting transformations without testing them.

A practical reading exercise: start with an unchanged baseline, add one transformation, and compare the same validation metric on the same splits. Fit learned transformations only on training folds. Use our feature-selection guide to distinguish creating features from choosing existing ones.

Put the fundamentals into practice. Read the free Starter Kit excerpt on test sets to preview the €19 handbook. No email required.

Review of Feature Engineering for Machine Learning

You’ve probably heard the phrase a million times:

Data cleaning and transformation can take substantial project time, but the familiar 80/20 split is a rule of thumb, not a universal measurement. This book focuses on the representation of data before and during modeling.

Feature engineering is an important part of a machine-learning pipeline. This book treats representation choices as a topic in their own right, rather than only as preparation for fitting a model.

With this very practical book, you’ll learn techniques for extracting and creating features out of raw data, so that your machine learning models can better understand them and therefore achieve better results.

Feature engineering for Machine learning covers the main techniques of today’s Data Science toolkit, as well as some hints on how to optimally apply them. The book uses the best Python packages including numpy, Pandas, Scikit-learn, and Matplotlib, featuring them in the wide variety of examples it includes.

In the text you will learn about the following topics:

Feature engineering for numeric data: filtering, binning, scaling, log transforms, and power transforms.
Natural text techniques: bag-of-words, n-grams, and phrase detection (check out our awesome NLP posts)
Frequency-based filtering and feature scaling for eliminating uninformative features
Encoding techniques of categorical variables, including feature hashing and bin-counting
Model-based feature engineering with Principal Component Analysis
The concept of model stacking, using k-means as a featurization technique
Image feature extraction with manual and deep-learning techniques

This book tries to teach the intuition first, and the mathematics second. Instead of only discussing how something is done, it tries to teach the why, with the goal of providing the intuition behind the ideas, so that the reader may understand how and when to apply them.

Contents of Feature Engineering for Machine Learning

The contents of the book are the following:

The Machine Learning Pipeline: Data, Tasks, Models and Features.
Fancy Tricks with Simple Numbers: scalars, vectors, binarization, log transforms, feature scaling and normalization.
Text Data: Flattening, Filtering, and Chunking: N-grams and bag of words.
The Effects of Feature Scaling: From Bag-of-Words to Tf-Idf
Categorical Variables: Counting Eggs in the Age of Robotic Chickens: one hot and dummies, categorical variable encoding types, and feature hashing.
Dimensionality Reduction: Squashing the Data Pancake with PCA: PCA, its uses, advantages, and limitations.
Nonlinear Featurization via K-Means Model Stacking: K means clustering and how to use the results as features for machine learning models.
Automating the Featurizer: Image Feature Extraction and Deep Learning: a full chapter on how to extract awesome features out of our image data.
Back to the Feature: Building an Academic Paper Recommender: a very interesting final project to finish the book with.

You can find the official website of the book here.

Summary of Feature Engineering for Machine Learning

This is a very easy to read book that covers an essential and many times overseen part of a Machine Learning Pipeline. It contains very clear Python coding examples, and we suggest that you try them yourself and play around with the code, exploring what you can do with it.

Try one chapter in Python: use our MinMaxScaler example, compare scaled and unscaled inputs with a scale-sensitive model, and explain the validation result in your notebook.

It is pretty short and can be digested fairly quickly, so we recommend reading it after books like Python Machine Learning or Hands on Machine Learning with Python, Tensorflow and Keras. You best have some Python knowledge when tackling Feature Engineering for Machine learning, so if you don’t we recommend texts like Python Crash Course or our awesome list of Python Programming online courses.

Once you’ve read it, you might want to further down your statistical knowledge with texts like Practical Statistics for Data Scientist or to start grasping some coding architecture patterns with amazing books like Architecture Patterns with Python. Check out our reviews!