How to solve SettingWithCopyWarning when using the ‘inplace’ parameter in pandas

The SettingWithCopyWarning message is a confusing warning to many who are new to Pandas. If you’ve ever taken a computer science course, you might be aware of passing/copying by value or by reference. Well, it very much applies to pandas DataFrames too. Let’s go.

Basically, when you are slicing a DataFrame, Pandas isn’t always very clear whether you are making a copy or a view. A copy is truly a new object, while a view is only a subset that refers to the initial object. When you apply changes to an object that might be a view, and not a DataFrame, you’ll run into the following error.

SettingWithCopyWarning:
A value is trying to be set on a copy of a slice from a DataFrame.

If this message occurs when you’re using the inplace parameter, you’re likely applying a transformation on a subset of another DataFrame. I created an apple-themed example below (the fruit, not the tech giant).

apples = pd.DataFrame({
  'type': ['jonagold', 'golden', 'granny smith'], 
  'weight': [100, 120, 110]
})

apples2 = apples[apples['weight'] >= 110]

apples2['type'].replace({'golden': 'cox'}, inplace=True)

When you use the inplace parameter in the replace method, you are applying the changes to apples2, which is a view of apples.

But… everything works as expected. We changed apples2 as we wanted, and apples is untouched. Then why is Python showing the error? Because we are modifying a view, Python can’t know if it is our intention to edit the initial DataFrame apples, or the view apples2.

You can check if a DataFrame is a copy or a view as follows:

apples2._is_view

To avoid the error, it is good practice to copy() a DataFrame when slicing/subsetting it. I modified the initial example:

apples = pd.DataFrame({
  'type': ['jonagold', 'golden', 'granny smith'], 
  'weight': [100, 120, 110]
})

apples2 = apples[apples['weight'] >= 110].copy()
apples2['type'].replace({'golden': 'cox'}, inplace=True)

By the way, I didn’t necessarily come up with this solution myself. Although I’m grateful you’ve visited this blog post, you should know I get a lot from websites like StackOverflow and I have a lot of coding books. This one by Matt Harrison (on Pandas 1.x!) has been updated in 2020 and is an absolute primer on Pandas basics. If you want something broad, ranging from data wrangling to machine learning, try “Mastering Pandas” by Stefanie Molin.

Great success!