Data preparation and EDA

R PROGRAMMING • DATA ANALYSIS PRACTICAL

Statistics, Data Pre-processing
and PCA Using R

A practical guide to descriptive statistics, handling missing values, Min–Max normalization and dimensionality reduction using Principal Component Analysis on a Houses dataset.

Describe → Clean → Normalize → Reduce Dimensions → Interpret

Why these operations belong in one workflow

Statistical summaries describe a dataset, pre-processing improves its quality and comparability, and PCA transforms correlated variables into a smaller set of components.

Mean, median, mode and standard deviation reveal different aspects of a variable. Missing-value treatment prevents incomplete records from silently distorting calculations. Min–Max normalization places variables with different units on a comparable range. PCA then identifies directions that preserve as much variation as possible while reducing the number of dimensions.

Central principle: dimensionality reduction should be applied only after the variables have been inspected and appropriately pre-processed. PCA cannot repair incorrect values, unsuitable encodings or unjustified missing-data treatment.

Download the Houses dataset

The dataset contains 30 houses and ten columns. Four predictor values are intentionally missing so that missing-data operations can be practised. Numerical variables include price, area, bedrooms, bathrooms, property age, distance, floors, parking and quality score.

Only the CSV dataset is downloadable from this article.

Save location: place houses_dataset.csv inside the current R working directory, or provide its complete file path in read.csv().

1. Import and inspect the data in R

# Check or change the working directory
getwd()
# setwd("D:/R_Practical")

# Import the dataset
houses <- read.csv(
  "houses_dataset.csv",
  stringsAsFactors = FALSE
)

# Initial inspection
head(houses)
tail(houses)
dim(houses)
names(houses)
str(houses)
summary(houses)

str() reveals how R interpreted each column. summary() combines distribution summaries with missing-value counts. Before calculating statistics, verify that numerical variables are stored as numeric or integer values and that House_ID remains an identifier.

2. Mean, median, mode and standard deviation

Measures of central tendency locate a typical value, whereas standard deviation describes dispersion. No single statistic is sufficient for every distribution.

Statistic	Meaning	When it is useful	Sensitivity
Mean	Arithmetic average	Approximately symmetric numerical data	Strongly affected by extreme values
Median	Middle ordered value	Skewed numerical data such as house prices	More resistant to extreme values
Mode	Most frequent value	Categorical or discrete variables such as bedrooms	May be absent or have multiple values
Standard deviation	Typical spread around the mean	Comparing variability in a common unit	Affected by outliers and scale

Mean

Mean = (x₁ + x₂ + ··· + xₙ) ÷ n

mean_price <- mean(houses$Price_Lakh, na.rm = TRUE)
mean_area  <- mean(houses$Area_sqft, na.rm = TRUE)

print(mean_price)
print(mean_area)

The argument na.rm = TRUE removes missing observations for that calculation. Without it, the result becomes NA when the vector contains a missing value.

Median

median_price <- median(houses$Price_Lakh, na.rm = TRUE)
median_age   <- median(houses$Age_years, na.rm = TRUE)

print(median_price)
print(median_age)

The median divides the ordered observations into two halves. In housing data, a small number of very expensive properties may raise the mean more strongly than the median.

Mode

Base R's mode() reports an object's storage mode rather than the most frequent statistical value. Therefore, define a statistical-mode function:

stat_mode <- function(x) {
  x <- x[!is.na(x)]
  frequencies <- table(x)
  modes <- names(frequencies)[
    frequencies == max(frequencies)
  ]
  return(type.convert(modes, as.is = TRUE))
}

mode_bedrooms <- stat_mode(houses$Bedrooms)
mode_parking  <- stat_mode(houses$Parking)

print(mode_bedrooms)
print(mode_parking)

The function returns every tied mode. This is important because a variable may be bimodal or multimodal.

Standard deviation

Sample SD = √[ Σ(xᵢ − x̄)² ÷ (n − 1) ]

sd_price    <- sd(houses$Price_Lakh, na.rm = TRUE)
sd_area     <- sd(houses$Area_sqft, na.rm = TRUE)
sd_distance <- sd(houses$Distance_km, na.rm = TRUE)

print(sd_price)
print(sd_area)
print(sd_distance)

R's sd() calculates sample standard deviation using n − 1 in the denominator. Comparing raw standard deviations across variables measured in lakh, square feet and kilometres is usually not meaningful because the units and scales differ.

Generate a compact statistical summary

price_statistics <- c(
  Price_Mean = mean(houses$Price_Lakh, na.rm = TRUE),
  Price_Median = median(houses$Price_Lakh, na.rm = TRUE),
  Bedroom_Mode = stat_mode(houses$Bedrooms)[1],
  Price_Standard_Deviation = sd(
    houses$Price_Lakh,
    na.rm = TRUE
  )
)

round(price_statistics, 2)

3. Handling missing data

Missing values may arise from non-response, recording errors, unavailable measurements or data-integration problems. The reason matters because each treatment makes assumptions about the unobserved value.

Detect and count missing values

# Total missing values
sum(is.na(houses))

# Missing values by column
colSums(is.na(houses))

# Rows containing at least one missing value
houses[!complete.cases(houses), ]

Option A: remove incomplete rows

houses_complete <- na.omit(houses)

dim(houses)
dim(houses_complete)

Deletion is simple but reduces the sample size. It is defensible only when the missing portion is small and the missingness mechanism does not introduce systematic bias.

Option B: numerical imputation

# Median imputation is robust to skewness
houses$Area_sqft[
  is.na(houses$Area_sqft)
] <- median(
  houses$Area_sqft,
  na.rm = TRUE
)

# Mode imputation for a discrete variable
houses$Bedrooms[
  is.na(houses$Bedrooms)
] <- stat_mode(houses$Bedrooms)[1]

# Median imputation for age and distance
houses$Age_years[
  is.na(houses$Age_years)
] <- median(houses$Age_years, na.rm = TRUE)

houses$Distance_km[
  is.na(houses$Distance_km)
] <- median(houses$Distance_km, na.rm = TRUE)

# Confirm that missing values are resolved
colSums(is.na(houses))

Important: mean, median and mode imputation reduce apparent variability and can weaken relationships among variables. For research-grade analysis, consider the missingness mechanism and methods such as multiple imputation.

4. Min–Max normalization

House area may range in hundreds or thousands of square feet, while parking may contain only 0, 1 or 2. Min–Max normalization maps each variable into a common interval, normally 0 to 1.

x′ = (x − minimum) ÷ (maximum − minimum)

The minimum becomes 0, the maximum becomes 1, and all other observations lie proportionally between them.

Normalize one variable

min_max <- function(x) {
  range_x <- max(x, na.rm = TRUE) -
             min(x, na.rm = TRUE)

  if (range_x == 0) {
    return(rep(0, length(x)))
  }

  (x - min(x, na.rm = TRUE)) / range_x
}

houses$Area_Normalized <- min_max(
  houses$Area_sqft
)

head(houses[c("Area_sqft", "Area_Normalized")])

The zero-range condition prevents division by zero when every observation has the same value.

Normalize all PCA predictor variables

pca_columns <- c(
  "Area_sqft",
  "Bedrooms",
  "Bathrooms",
  "Age_years",
  "Distance_km",
  "Floors",
  "Parking",
  "Quality_Score"
)

normalized_houses <- as.data.frame(
  lapply(
    houses[pca_columns],
    min_max
  )
)

summary(normalized_houses)

Limitation: Min–Max normalization is sensitive to extreme values because the minimum and maximum determine the scale. New values outside the original range can also produce normalized values below 0 or above 1.

5. Principal Component Analysis for Houses data

PCA converts correlated numerical variables into new uncorrelated variables called principal components. The first component captures the greatest possible variance, the second captures the greatest remaining variance subject to being orthogonal to the first, and the process continues for later components.

Original variablesArea, bedrooms, bathrooms, age, distance, floors, parking and quality may overlap in the information they contain.

Principal componentsEach component is a weighted linear combination of the original variables.

ScoresCoordinates of individual houses in the new component space.

LoadingsWeights showing how strongly each original variable contributes to a component.

Step 1: verify PCA input

# PCA requires complete numerical data
stopifnot(
  all(sapply(normalized_houses, is.numeric)),
  sum(is.na(normalized_houses)) == 0
)

# Check for zero-variance variables
variances <- sapply(normalized_houses, var)
print(variances)

Step 2: calculate PCA

# Data are already Min-Max normalized.
# Centre before PCA but do not scale a second time.
pca_model <- prcomp(
  normalized_houses,
  center = TRUE,
  scale. = FALSE
)

summary(pca_model)

summary() reports the standard deviation of each principal component, its proportion of variance and cumulative variance. A common practical aim is to retain enough components to explain a chosen proportion such as 80% or 90%, but the threshold should follow the analytical purpose.

Step 3: examine loadings

loadings <- pca_model$rotation
round(loadings, 3)

Large positive or negative loading magnitudes indicate variables that contribute strongly to a component. The sign shows direction, but the sign of an entire principal component can be reversed without changing the solution's meaning.

Step 4: examine component scores

scores <- as.data.frame(pca_model$x)

pca_result <- cbind(
  House_ID = houses$House_ID,
  scores
)

head(pca_result)

Step 5: scree plot and cumulative variance

# Built-in scree plot
screeplot(
  pca_model,
  type = "lines",
  main = "Scree Plot for Houses Dataset"
)

# Variance explained
variance_ratio <- (
  pca_model$sdev^2 /
  sum(pca_model$sdev^2)
)

cumulative_variance <- cumsum(variance_ratio)

plot(
  cumulative_variance,
  type = "b",
  ylim = c(0, 1),
  xlab = "Number of Principal Components",
  ylab = "Cumulative Variance Explained",
  main = "Cumulative Variance Explained"
)
abline(h = 0.80, col = "red", lty = 2)

Step 6: PCA biplot

biplot(
  pca_model,
  scale = 0,
  cex = 0.7,
  main = "PCA Biplot: Houses Dataset"
)

Points close together represent houses with similar standardized profiles. Variable arrows pointing in similar directions suggest positive association; arrows pointing in opposite directions suggest negative association. Longer arrows indicate variables represented more strongly in the displayed component plane.

Methodological note: many PCA studies use z-score standardization rather than Min–Max normalization. To use that approach, run prcomp(houses[pca_columns], center = TRUE, scale. = TRUE) after imputation. Do not combine Min–Max normalization with scale. = TRUE unless that two-stage transformation is deliberately required.

6. Complete executable R program

# -------------------------------------------------
# STATISTICS, PRE-PROCESSING AND PCA USING R
# -------------------------------------------------

# 1. Import
houses <- read.csv(
  "houses_dataset.csv",
  stringsAsFactors = FALSE
)

# 2. Statistical mode function
stat_mode <- function(x) {
  x <- x[!is.na(x)]
  frequencies <- table(x)
  modes <- names(frequencies)[
    frequencies == max(frequencies)
  ]
  type.convert(modes, as.is = TRUE)
}

# 3. Descriptive statistics
statistics <- c(
  Price_Mean = mean(houses$Price_Lakh, na.rm = TRUE),
  Price_Median = median(houses$Price_Lakh, na.rm = TRUE),
  Bedroom_Mode = stat_mode(houses$Bedrooms)[1],
  Price_SD = sd(houses$Price_Lakh, na.rm = TRUE)
)
print(round(statistics, 2))

# 4. Missing-value report
print(colSums(is.na(houses)))

# 5. Imputation
houses$Area_sqft[is.na(houses$Area_sqft)] <-
  median(houses$Area_sqft, na.rm = TRUE)

houses$Bedrooms[is.na(houses$Bedrooms)] <-
  stat_mode(houses$Bedrooms)[1]

houses$Age_years[is.na(houses$Age_years)] <-
  median(houses$Age_years, na.rm = TRUE)

houses$Distance_km[is.na(houses$Distance_km)] <-
  median(houses$Distance_km, na.rm = TRUE)

# 6. Min-Max function
min_max <- function(x) {
  range_x <- max(x) - min(x)
  if (range_x == 0) {
    return(rep(0, length(x)))
  }
  (x - min(x)) / range_x
}

# 7. Select and normalize PCA variables
pca_columns <- c(
  "Area_sqft", "Bedrooms", "Bathrooms",
  "Age_years", "Distance_km", "Floors",
  "Parking", "Quality_Score"
)

normalized_houses <- as.data.frame(
  lapply(houses[pca_columns], min_max)
)

# 8. PCA
pca_model <- prcomp(
  normalized_houses,
  center = TRUE,
  scale. = FALSE
)

# 9. Results
print(summary(pca_model))
print(round(pca_model$rotation, 3))

# 10. Plots
screeplot(pca_model, type = "lines")
biplot(pca_model, scale = 0, cex = 0.7)

Interpretation checklist

1. Compare mean and median to identify possible skewness.
2. Report the mode only with its variable and frequency context.
3. State whether standard deviation is based on a sample.
4. Count missing values before and after treatment.
5. Justify deletion or imputation instead of applying it automatically.
6. Verify the normalized minimum and maximum values.
7. Exclude identifiers and unsuitable categorical fields from PCA.
8. Check for missing and zero-variance variables before PCA.
9. Report variance explained and the component-retention rule.
10. Interpret loadings using domain meaning, not magnitude alone.

Practice questions

1. Calculate mean, median, mode and standard deviation for Area_sqft.
2. Compare the mean and median house prices. What does the difference suggest?
3. Replace median imputation with mean imputation and compare the results.
4. Normalize Price_Lakh separately and verify its minimum and maximum.
5. Run PCA using z-score standardization and compare variance explained with the Min–Max version.
6. Identify the variables with the three largest absolute loadings on PC1.
7. Determine the minimum number of components needed to explain at least 80% of total variance.
8. Create a scatter plot of PC1 against PC2 and label houses using House_ID.

R Practical • Statistical Operations • Data Pre-processing • PCA