Tabular Data Classification

Project

Table of contents

• Overview
• Details of Dataset
• Data Preparation
• Architecture of Network
• Training and Test
• Evaluation
• Additional Sections
• Conclusion
• Resources
• Project repository

---

Overview

This project demonstrates how to implement, train, and evaluate a neural network for tabular data classification using PyTorch. The workflow includes data loading, preprocessing, model definition, training, evaluation, and prediction.

---

Details of Dataset

• Dataset Used: BUPA Liver Disorders Dataset
• Source: UCI Machine Learning Repository - Liver Disorders
• Description: The dataset contains 6 attributes for each patient, with the goal of classifying liver disorders.
• Number of Features: 6
• Task: Binary classification (liver disorder or not)

# Dataset class for CSV data
from torch.utils.data import Dataset, random_split
from pandas import read_csv

class CSVDataset(Dataset):
  def __init__(self, path):
    df = read_csv(path, header=None)
    self.X = df.values[:, :-1].astype('float32')
    self.y = df.values[:, -1].astype('float32') - 1
    self.y = self.y.reshape((len(self.y), 1))
  def __len__(self):
    return len(self.X)
  def __getitem__(self, idx):
    return [self.X[idx], self.y[idx]]
  def get_splits(self, n_test=0.2):
    test_size = round(n_test * len(self.X))
    train_size = len(self.X) - test_size
    return random_split(self, [train_size, test_size])

Explanation of the CSVDataset class and related code:

This code defines a custom PyTorch Dataset for loading tabular data from a CSV file:

• from torch.utils.data import Dataset, random_split: Imports PyTorch’s Dataset base class and a utility to split datasets into training and test sets.
• from pandas import read_csv: Imports pandas for reading CSV files.
• class CSVDataset(Dataset): Subclasses Dataset to create a custom dataset for tabular data.
  • The __init__ method loads the CSV file, extracts features (self.X) and labels (self.y), converts them to float32, and reshapes the labels for compatibility with PyTorch models. Optionally, normalization and label encoding can be applied (commented in the code).
  • The __len__ method returns the number of samples in the dataset.
  • The __getitem__ method retrieves a single sample (features and label) by index.
  • The get_splits method splits the dataset into training and test sets using a specified test fraction (default 20%). This class makes it easy to load, preprocess, and split tabular data for deep learning workflows in PyTorch.

---

Data Preparation

# Data loader preparation
from torch.utils.data import DataLoader

def prepare_data(path):
  dataset = CSVDataset(path)
  train, test = dataset.get_splits()
  train_dl = DataLoader(train, batch_size=32, shuffle=True)
  test_dl = DataLoader(test, batch_size=1024, shuffle=False)
  return train_dl, test_dl

Architecture of Network

Defining neural network:
A simple Multi-Layer Perceptron (MLP) is used, implemented as a subclass of torch.nn.Module. This structure is simple and effective for small tabular datasets. The hidden layers allow the model to learn non-linear relationships between features.

Architecture of the used MLP

- Input layer: 6 features
- Hidden layer 1: 10 units, ReLU activation
- Hidden layer 2: 8 units, ReLU activation
- Output layer: 1 unit, Sigmoid activation (for binary classification)
- Weight initialization: Kaiming for hidden layers, Xavier for output

import torch.nn as nn
import torch.nn.functional as F
from torch.nn.init import kaiming_uniform_, xavier_uniform_

class MLP(nn.Module):
  def __init__(self, n_inputs):
    super(MLP, self).__init__()
    self.hidden1 = nn.Linear(n_inputs, 10)
    kaiming_uniform_(self.hidden1.weight, nonlinearity='relu')
    self.act1 = nn.ReLU()
    self.hidden2 = nn.Linear(10, 8)
    kaiming_uniform_(self.hidden2.weight, nonlinearity='relu')
    self.act2 = nn.ReLU()
    self.hidden3 = nn.Linear(8, 1)
    xavier_uniform_(self.hidden3.weight)
    self.act3 = nn.Sigmoid()
  def forward(self, X):
    X = self.hidden1(X)
    X = self.act1(X)
    X = self.hidden2(X)
    X = self.act2(X)
    X = self.hidden3(X)
    X = self.act3(X)
    return X

---

Training and Test

Loss function
Binary Cross Entropy Loss (BCELoss) is used, which is standard for binary classification. It measures the difference between predicted probabilities and true labels.

Optimizer
SGD optimizer (torch.optim.SGD) is used with momentum for efficient convergence.

Settings

• Batch size: 32 (train), 1024 (test)
• Learning rate: 0.01
• Epochs: 100

from torch.optim import SGD
from torch.nn import BCELoss

def train_model(train_dl, model):
  criterion = BCELoss()
  optimizer = SGD(model.parameters(), lr=0.01, momentum=0.9)
  for epoch in range(100):
    for i, (inputs, targets) in enumerate(train_dl):
      optimizer.zero_grad()
      yhat = model(inputs)
      loss = criterion(yhat, targets)
      loss.backward()
      optimizer.step()

---

Evaluation

• Accuracy is calculated using sklearn.metrics.accuracy_score.
• Predictions are rounded to 0 or 1 for binary classification.

from numpy import vstack
from sklearn.metrics import accuracy_score
from torch import Tensor

def evaluate_model(test_dl, model):
  predictions, actuals = list(), list()
  for i, (inputs, targets) in enumerate(test_dl):
    yhat = model(inputs)
    yhat = yhat.detach().numpy()
    actual = targets.numpy()
    actual = actual.reshape((len(actual), 1))
    yhat = yhat.round()
    predictions.append(yhat)
    actuals.append(actual)
  predictions, actuals = vstack(predictions), vstack(actuals)
  acc = accuracy_score(actuals, predictions)
  return acc

# Single prediction example
def predict(row, model):
  row = Tensor([row])
  yhat = model(row)
  yhat = yhat.detach().numpy()
  return yhat

---

Additional Sections

Saving the Model:
The trained model can be saved using:

torch.save(model.state_dict(), "mlp_model.pth")

Hardware Utilization:
The code can be adapted to use GPU if available for faster training.

Improvement Suggestions:

• Add normalization to input features
• Experiment with different architectures, optimizers, and learning rates
• Use regularization (e.g., weight decay)
• Try advanced optimizers like Adam or AdamW

---

Conclusion

This project covers the essential steps for building a tabular data classifier using PyTorch:

• Data loading and normalization
• Model architecture design
• Training and evaluation
• Model saving and improvement

You can extend this project by experimenting with different architectures, datasets, and hyperparameters.

---

Resources

• UCI Machine Learning Repository - Liver Disorders
• PyTorch Documentation
• pandas Documentation
• scikit-learn Documentation
• McMedHacks

Project repository

GitHub Code: Medical Image Classification, Segmentation using U-Net, and Tabular Data Classification