Perform Data Analysis Using Python

Python provides a practical set of libraries for importing, cleaning, exploring, visualizing, and summarizing data. Beginners can start with core Python, NumPy, pandas, and Matplotlib, then add other tools when a project requires statistical analysis, machine learning, database access, or automated data pipelines.

This tutorial explains how to learn Python for data analysis and demonstrates a complete workflow with a small dataset. The examples cover loading data, checking its structure, handling missing values, calculating summary statistics, grouping records, and creating a chart.

Python Data Analysis Workflow for Beginners

A typical Python data analysis project follows these seven steps:

1. Define the question that the analysis must answer.
2. Collect data from files, databases, APIs, surveys, or applications.
3. Inspect the data structure, column types, ranges, and missing values.
4. Clean incorrect, incomplete, duplicated, or inconsistent records.
5. Transform and organize the data for analysis.
6. Analyze and visualize patterns, relationships, and differences.
7. Interpret the results and communicate conclusions with appropriate context.

These steps are iterative. An unexpected result may require returning to the cleaning or transformation stage before drawing a conclusion.

Python Tools Needed for Data Analysis

You do not need to learn every Python library before starting. The following tools cover most beginner and intermediate data analysis tasks.

Tool	Role in data analysis	Topics to learn first
Python	Controls the analysis workflow and provides general programming features	Variables, lists, dictionaries, conditions, loops, functions, and exceptions
Jupyter Notebook	Combines executable code, output, charts, and written explanations	Cells, kernel execution, Markdown, and notebook files
NumPy	Provides efficient multidimensional arrays and numerical operations	Arrays, shapes, indexing, slicing, vectorized operations, and aggregations
pandas	Provides DataFrame and Series structures for tabular data	Reading files, selecting rows, missing values, grouping, merging, and sorting
Matplotlib	Creates charts for exploratory analysis and reporting	Line charts, bar charts, scatter plots, labels, legends, and figure sizing

Core Python Concepts for Data Analysis

Begin with the parts of Python that are used repeatedly in analysis scripts. You should be able to store values, work with collections, write conditions, repeat operations, and organize reusable logic in functions.

• Numbers, strings, Boolean values, and None
• Lists, tuples, sets, and dictionaries
• if, elif, and else conditions
• for and while loops
• Functions, parameters, return values, and scope
• File handling and exception handling
• Importing modules and installing packages

You do not need advanced object-oriented programming before analyzing your first dataset. Learn additional language features as your projects become more complex.

NumPy Arrays for Numerical Data

NumPy is the foundation of many scientific Python libraries. It provides an array structure for storing numerical data and performing calculations without writing a separate Python loop for every value.

import numpy as np

sales = np.array([1200, 1450, 980, 1675, 1530])

print("Total:", sales.sum())
print("Average:", sales.mean())
print("Highest:", sales.max())
print("Above average:", sales[sales > sales.mean()])

Total: 6835
Average: 1367.0
Highest: 1675
Above average: [1450 1675 1530]

Important NumPy topics include array creation, dimensions, shapes, data types, indexing, slicing, Boolean filtering, broadcasting, aggregation functions, and missing-value handling with appropriate numerical techniques.

pandas DataFrames for Tabular Analysis

pandas is commonly used for data arranged in rows and columns. A DataFrame represents a table, while a Series represents one labeled column of data.

Learn how to read CSV and Excel files, inspect columns, filter rows, clean missing values, group records, combine tables, create calculated columns, and export results.

import pandas as pd

sales_data = {
    "region": ["North", "South", "North", "West", "South"],
    "product": ["Laptop", "Laptop", "Tablet", "Tablet", "Laptop"],
    "units": [5, 7, 4, 6, 3],
    "unit_price": [800, 800, 450, 450, 800]
}

df = pd.DataFrame(sales_data)
df["revenue"] = df["units"] * df["unit_price"]

print(df)
print(df.groupby("region")["revenue"].sum())

The calculated revenue column is created from two existing columns. The groupby() operation then calculates total revenue for each region.

Matplotlib Charts for Exploring Data

Matplotlib can be used to create line charts, bar charts, histograms, scatter plots, box plots, and other visualizations. A chart should help answer a defined question rather than simply decorate the analysis.

import matplotlib.pyplot as plt

revenue_by_region = df.groupby("region")["revenue"].sum().sort_values()

revenue_by_region.plot(kind="bar")
plt.title("Revenue by Region")
plt.xlabel("Region")
plt.ylabel("Revenue")
plt.tight_layout()
plt.show()

When learning Matplotlib, focus on selecting an appropriate chart type, adding meaningful axis labels, controlling figure size, formatting scales, and avoiding visual elements that can misrepresent the data.

Install Python Data Analysis Libraries

Create a separate virtual environment for each project so that package versions and dependencies remain isolated.

python -m venv data-analysis-env

# Activate on Windows
data-analysis-env\Scripts\activate

# Activate on macOS or Linux
source data-analysis-env/bin/activate

python -m pip install numpy pandas matplotlib jupyter

Start Jupyter Notebook from the activated environment:

jupyter notebook

You can also run the examples in a Python script or an integrated development environment. Jupyter is useful during exploration because code, notes, tables, and charts can be kept together.

Perform Data Analysis Using Python: Complete Example

The following example analyzes a CSV file containing order records. Assume the file is named orders.csv and contains these columns:

order_id,order_date,region,product,units,unit_price
1001,2026-01-03,North,Laptop,2,800
1002,2026-01-04,South,Tablet,3,450
1003,2026-01-05,North,Tablet,,450
1004,2026-01-06,West,Laptop,1,800
1005,2026-01-07,South,Laptop,4,800

Load and Inspect the Python Dataset

import pandas as pd

orders = pd.read_csv("orders.csv")

print(orders.head())
print(orders.shape)
print(orders.dtypes)
print(orders.isna().sum())

The first inspection should answer four questions: what columns are present, how many rows exist, which data types pandas inferred, and where values are missing.

Clean Missing and Incorrect Values in pandas

orders["order_date"] = pd.to_datetime(
    orders["order_date"],
    errors="coerce"
)

orders["units"] = pd.to_numeric(
    orders["units"],
    errors="coerce"
)

orders = orders.dropna(
    subset=["order_date", "region", "product", "unit_price"]
)

orders["units"] = orders["units"].fillna(0)
orders = orders.drop_duplicates(subset="order_id")

The correct treatment of missing values depends on the dataset. Replacing missing units with zero is suitable only when zero accurately represents the business meaning. In another dataset, removing the row, using a statistical estimate, or requesting the original value may be more appropriate.

Create Calculated Columns for Python Analysis

orders["revenue"] = orders["units"] * orders["unit_price"]
orders["order_month"] = orders["order_date"].dt.to_period("M")

print(orders[[
    "order_id",
    "region",
    "product",
    "units",
    "revenue"
]])

Calculated columns convert raw values into measures needed by the analysis. Keep the calculation explicit so another person can verify how each result was produced.

Calculate Descriptive Statistics with pandas

print(orders[["units", "unit_price", "revenue"]].describe())

print("Total revenue:", orders["revenue"].sum())
print("Average order revenue:", orders["revenue"].mean())
print("Median order revenue:", orders["revenue"].median())

Descriptive statistics summarize the data but do not explain why a pattern occurred. Consider sample size, missing records, outliers, time periods, and collection methods before interpreting the values.

Group Python Data by Region and Product

regional_summary = (
    orders.groupby("region", as_index=False)
    .agg(
        total_units=("units", "sum"),
        total_revenue=("revenue", "sum"),
        order_count=("order_id", "count")
    )
    .sort_values("total_revenue", ascending=False)
)

product_summary = (
    orders.groupby("product", as_index=False)["revenue"]
    .sum()
    .sort_values("revenue", ascending=False)
)

print(regional_summary)
print(product_summary)

Named aggregations make the resulting columns easier to understand. The analysis now shows total units, revenue, and order count for each region.

Visualize the Python Analysis Result

import matplotlib.pyplot as plt

regional_summary.plot(
    x="region",
    y="total_revenue",
    kind="bar",
    legend=False
)

plt.title("Total Revenue by Region")
plt.xlabel("Region")
plt.ylabel("Revenue")
plt.xticks(rotation=0)
plt.tight_layout()
plt.show()

A bar chart is appropriate here because the analysis compares one numerical measure across discrete categories. A line chart would be more suitable for a value measured continuously over time.

Export the Cleaned Python Analysis

orders.to_csv("cleaned_orders.csv", index=False)
regional_summary.to_csv("regional_summary.csv", index=False)

Exporting both the cleaned data and the summarized result makes the workflow easier to review. Keep the original source file unchanged so that the analysis can be reproduced from the starting data.

Four Types of Data Analysis in Python

Data analysis is often divided into four broad types. The distinction is based on the question being asked rather than the Python library being used.

Analysis type	Question answered	Python example
Descriptive analysis	What happened?	Calculate monthly revenue, average order value, or customer counts
Diagnostic analysis	Why did it happen?	Compare groups, examine correlations, and investigate contributing factors
Predictive analysis	What may happen next?	Build a model to estimate future sales or classify outcomes
Prescriptive analysis	What action should be taken?	Evaluate alternatives under constraints and recommend an action

Predictive and prescriptive work requires more than running a model. The assumptions, training data, uncertainty, evaluation method, and operational constraints must also be examined.

Python Data Analysis and ETL Workflows

Python is a programming language, not a single ETL product. However, it can be used to build extract, transform, and load workflows.

• Extract: Read data from CSV files, spreadsheets, databases, APIs, or cloud storage.
• Transform: Clean values, combine tables, standardize formats, validate records, and calculate new fields.
• Load: Write the processed data to a database, data warehouse, file, or reporting system.

A small ETL task can be written with pandas and database libraries. Larger production pipelines usually require scheduling, monitoring, retries, logging, testing, access controls, and data-quality checks in addition to transformation code.

Python Data Analysis Learning Path

A structured learning sequence is more effective than attempting to study many libraries at the same time.

1. Learn basic Python syntax and write small programs.
2. Practice reading and writing CSV, JSON, and text files.
3. Learn NumPy arrays, indexing, filtering, and aggregation.
4. Learn pandas DataFrames, cleaning, grouping, merging, and reshaping.
5. Create common charts with Matplotlib.
6. Study descriptive statistics and basic probability.
7. Complete projects using real datasets with documented assumptions.
8. Learn SQL because many analysis datasets are stored in relational databases.
9. Add statistical modeling or machine learning only when it supports the question being studied.

The time required depends on prior experience, practice frequency, and the complexity of the analysis. A beginner may understand basic Python syntax within several days, but dependable data analysis requires continued practice with cleaning, validation, statistics, and interpretation.

Common Python Data Analysis Mistakes

• Starting without a clear question: Define the decision or problem before selecting calculations and charts.
• Ignoring data types: Dates stored as text and numbers stored as strings can produce incorrect operations.
• Removing missing values without review: Missingness may contain useful information or introduce bias.
• Changing the original dataset: Preserve a raw copy and perform cleaning in a reproducible script.
• Using averages alone: Check distributions, medians, ranges, sample sizes, and outliers.
• Treating correlation as proof of causation: A relationship between variables does not by itself establish cause.
• Creating misleading charts: Use appropriate scales, labels, units, and chart types.
• Skipping validation: Compare totals, row counts, and selected records before trusting the result.
• Learning too many libraries at once: Develop a reliable workflow with a small toolset before adding specialized packages.

Python Data Analysis Editorial QA Checklist

• Confirm that every code example imports the libraries it uses.
• Verify that CSV column names match the names referenced in the pandas code.
• Check that missing-value decisions are explained instead of applied without context.
• Confirm that calculated columns use the intended units and formulas.
• Verify grouped totals against a manual calculation or an independent query.
• Check that each chart type matches the analytical question.
• Confirm that chart axes, labels, categories, and units are clear.
• Distinguish observations in the data from assumptions or causal conclusions.
• Keep the original dataset separate from cleaned and summarized output files.
• Run the complete workflow from a fresh environment to confirm reproducibility.

Frequently Asked Questions About Data Analysis Using Python

Which Python libraries should a beginner learn for data analysis?

Start with NumPy for numerical arrays, pandas for tabular data, and Matplotlib for charts. Learn core Python and basic statistics alongside these libraries. Additional packages can be added when a project has a specific requirement.

Can Python be learned for data analysis in seven days?

A beginner can learn basic syntax and complete small guided exercises in seven days. Proficiency in cleaning unfamiliar datasets, choosing valid methods, detecting errors, and interpreting results takes longer and requires repeated project work.

What are the four types of data analysis?

The four commonly described types are descriptive, diagnostic, predictive, and prescriptive analysis. They address what happened, why it happened, what may happen next, and what action may be appropriate.

Is Python an ETL tool?

Python is a general-purpose programming language. It can be used to create ETL processes that extract, transform, and load data, but production ETL systems usually also need orchestration, monitoring, security, logging, and failure recovery.

Should I learn SQL before performing data analysis with Python?

You can begin with files and pandas before learning SQL, but SQL should be added early. Analysts frequently use SQL to retrieve, filter, join, and aggregate data before continuing the analysis in Python.