SyncAI.news, a Varaisys broadcasting
Why Most Data Science Notebooks Die After Day One: How to Build Ones That Survive
NR

Nate Rosidi

· 2 min read

EngineeringKDnuggets

Why Most Data Science Notebooks Die After Day One: How to Build Ones That Survive

A notebook dies the moment "Restart Kernel and Run All" stops working.

Nobody notices for a week. The analysis was finished, the chart went into a deck, the file got pushed. Then someone asks where a number came from; you open the notebook, run it from the top, and cell 12 throws a KeyError on a column you renamed in cell 31 and deleted in cell 44. The output cells still show the old numbers, so the notebook looks fine while being unrunnable.

The habits that prevent this are cheap. We are going to apply all of them to one real dataset and keep the whole thing under 100 lines of Pandas.

The Data

In this article, we are using a table called olympics_athletes_events, used in this interview question.

olympics_athletes_events is one row per athlete per event, which is the detail that matters later. Its 352 rows cover 336 athletes across 15 Games and 167 events, so 11 athletes appear more than once and one appears 6 times. The medal column is filled for 120 rows, and a blank means that athlete did not win a medal in that event.

Here is a sample:

id name sex age height team noc year sport medal
3520 Guillermo J. Amparan M Mexico MEX 1924 Athletics
35394 Henry John Finchett M Great Britain GBR 1924 Gymnastics
21918 Georg Frederik Ahrensborg Clausen M 28.0 Denmark DEN 1924 Cycling
110345 Marinus Cornelis Dick Sigmond M 26.0 Netherlands NED 1924 Football
999998 John Testman M 30.0 180.0 Canada CAN 2004 Athletics Bronze

Let's now explore the habits that keep your notebook alive.

Habit 1: Adding Configurations to the First Cell

Every path, seed, threshold, and magic number goes in the first cell. Nothing else does.

Two important things happen here. Someone reading the notebook six months later can see every assumption in 15 lines without scrolling. And when the file moves or the threshold changes, there is exactly one place to edit.

Habit 2: Writing One Function per Cell

Habit 3: Validating the Data Before You Trust It

Running it on the raw file:

Original source

This story was published by KDnuggets and written by Nate Rosidi. SyncAI.news shows a preview; the complete article is on the publisher's site.

Read the full story on kdnuggets.com

Similar News