Data Science Process: Overview

Data Science Process: Overview

Data science process is a series of activities that help us move from a business problem to a useful solution based on data.

Define research goal

          ↓

Data retrieval

          ↓

Data preperation

          ↓

Data exploration

         ↓

Model building

          ↓

Presenting results and automation

Fig. 3.3.1 Data science process

Following are the steps of data science process ‒

1. Define research goals:

•  This is the first and most important step.

•  This is a step in which we clearly define what problem we are solving, why it is important, and how to solve it.

• The output of this step is usually a short document called project charter which explains project's purpose and plan.

•  For example ‒ A company wants to reduce customer churn (people leaving their service). So research goal could be "Predict which customers are likely to leave in next three months"

2. Data retrieval:

•  After defining the goal, the next step is to collect data from right sources.

• We need to find relevant and reliable data for analysis.

•  Data can be collected from company databases, public web sites, APIs, Sensors, Surveys.

• The collected data is usually raw. That means it may have errors, missing values or inconsistent formats.

•  For example ‒ The company collects customer information like ‒ name, age, purchase history and subscription duration from its database.

3. Data preparation :

• This is a step in which data is cleaned and transform the raw data in such a manner that it

can be used in models. This activity is also called as data preprocessing.

•   It includes ‒

■ Removing duplicates

■ Filling in missing values.

■ Combining data from different sources

■ Changing data types or formats.

• For example ‒ If one data set records "Yes/No" for purchase and another uses "1/0”, we can convert them into a common format.

4. Data exploration :

• In this step we try to understand the data deeply before modeling.

• During data exploration process, we look for patterns, trends and relationships among the variables.

• We use graphs, charts and statistics to visualize and describe data.

• This helps us to decide which type of model or analysis will work best.

•  For example ‒ We might discover that customers aged 18‒25 have a higher chances of leaving compared to other age groups.

5. Model building:

•  This is a data modelling step. In this step, we apply machine learning algorithms (like linear regression model, Decision trees or clustering).

• Sometimes simple models can perform better than the complex ones.

• We can combine multiple simple models to improve the results.

• For example ‒ Build a prediction model to estimate which customers are likely to leave next month based on their behaviour.

6. Presenting results and automation:

• This is a final step of data science process.

• In this step, we have to communicate our results clearly to decision makers.

• Make use of charts, dashboards, or reports to show how your findings can improve the business.

• If the process needs to be repeated regularly then we can automate the model so it runs automatically with new data.

•  For example ‒ Present a report showing that customers who use the app less than twice a week are more likely to leave ‒ and recommend strategies to re‒engage them.