How to solve pandas not showing all columns

As an R-native, switching to Python and Pandas has its little annoyances. In R, when you print a data frame, or its head, you get to see the whole shebang. In Pandas, this is not the case: rows and columns get truncated to keep the output manageable. However, there are some settings that you can tweak. Here we go.

To try out something new, I’m working on this Kaggle project. Simply loading in the data set and using .head() learns you nothing, because only two columns are displayed (the first and the last) and the fourteen columns in the middle are truncated.

train.head()

However, with a simple line of code, you can decide how many columns are actually shown. To set it to ten (for example), you can use the set_option function or the dotted-style notation.

pd.set_option('display.max_columns', 10)
pd.options.display.max_columns = 10

To set the option to show all columns, you pass it None.

pd.set_option('display.max_columns', None)
train.head()

If you want to reset the max_columns option, you can use the reset_option function.

pd.reset_option('display.max_columns')

Interesting to know is that the set_option function does a regex match. Simply passing it ‘max_colu’ will also do the trick. ‘max_col’ will not work because there’s also a max_colwidth options, and pandas doesn’t know which option to change.

pd.set_option('max_colu', 10)

Here are some other options that are frequently used:

• display.min_columns — how many columns to show when exceeding max_rows?
• display.max_rows
• display.min_rows
• display.expand_frame_repr — break up output or have it stretch across multiple pages?
• display.large_repr — show summary of data frame when exceeding max_columns
• display.max_colwidth — max character in column values
• display.precision — decimal spaces

By the way, I didn’t necessarily come up with this solution myself. Although I’m grateful you’ve visited this blog post, you should know I get a lot from websites like StackOverflow and I have a lot of coding books. This one by Matt Harrison (on Pandas 1.x!) has been updated in 2020 and is an absolute primer on Pandas basics. If you want something broad, ranging from data wrangling to machine learning, try “Mastering Pandas” by Stefanie Molin.

Good luck!