Control Point Deformation Network
Project
Table of Contents
- Overview
- Introduction
- Problem Definition
- Dataset Resources
- Preprocessing Steps
- Modules and Architectures
- Mathematical Formulation
- Evaluation Metrics
- Results
- Conclusion
- Project repository
Overview
This post introduces the Controlled Point Deformation Network (CPDNet), a deep learning framework for patient-specific prediction of postoperative 3D spine shape in Adolescent Idiopathic Scoliosis (AIS) surgery. CPDNet models the spine as a semi-rigid articulated structure using anatomical landmarks and clinical metadata, and learns both local deformations and global shape consistency via a novel rigid transformation loss. The approach combines geometric accuracy and anatomical plausibility, outperforming existing point-based methods. Clinically, CPDNet can serve as a foundation for surgical planning and visualization tools, helping surgeons and patients understand expected outcomes for different surgical strategies.
Introduction
Adolescent Idiopathic Scoliosis (AIS) is a complex three-dimensional deformation of the spine that often requires surgical correction in severe cases. During surgical planning, both surgeons and patients would greatly benefit from being able to visualize the expected postoperative spine shape before surgery.
While biomechanical simulators and machine-learning approaches have been explored, existing methods either:
- require detailed mechanical modeling of the spine, or
- provide non-personalized or coarse predictions, such as clustered outcome categories.
Predicting a patient-specific 3D postoperative spine shape remains a challenging problem. In this work, the problem is addressed as a 3D point-set deformation task, where each spine is represented by anatomical landmarks. The paper introduces a novel deep learning architecture—Controlled Point Deformation Network (CPDNet)—that explicitly learns global spine shape properties through inter-vertebral rigid transformations rather than treating points independently.
Problem Definition
The goal is to predict a patient’s postoperative 3D spine shape given:
- the preoperative 3D spine configuration, and
- selected clinical metadata.
Formally, the task is framed as learning a deformation between two ordered 3D point sets:
- Input: Preoperative spine landmarks
- Output: Postoperative spine landmarks
Each spine is modeled as a semi-rigid articulated structure, where neighboring vertebrae are constrained by anatomical relationships. Unlike prior point-based networks that learn point displacements independently, CPDNet enforces neighborhood and global shape consistency by learning rigid transformations (rotation + translation) between consecutive vertebrae.
Dataset Resources
The dataset consists of 99 AIS patients who underwent posterior spinal fusion surgery between 2009 and 2018 at Sainte-Justine University Hospital (Montreal, Canada).
Key characteristics:
| Aspect | Details |
|---|---|
| Curve type | Lenke 1A main thoracic scoliosis |
| Imaging | Preoperative and postoperative biplanar X-ray scans |
| Timepoints | ~6 months before and after surgery |
| Surgeons | 3 orthopedic surgeons (used as a proxy for surgical strategy) |
| 3D Representation | 18 vertebral levels (C7 to L5); 6 anatomical landmarks per vertebra; Total: 108 3D points per spine |
| Clinical metadata used | Gender, Age, Height, Weight, BMI, Operating surgeon (categorical) |
The dataset is split into 80% training, 10% validation, and 10% testing.
Preprocessing Steps
3D Reconstruction Spine landmarks are reconstructed from biplanar X-ray images using established articulated spine modeling techniques.
Landmark Extraction Six anatomical landmarks are extracted per vertebra to form an ordered 3D point set.
Global Registration All spine shapes are registered to a common coordinate system anchored at L5, ensuring spatial consistency across patients.
Inter-vertebral Transform Computation (Ground Truth) Rigid transformations (rotation + translation) between consecutive vertebrae are computed using articulated spine mathematics. These transformations serve as supervision for the proposed rigid transformation loss.
Modules and Architectures
Controlled Point Deformation Network (CPDNet) predicts point-wise displacement vectors while respecting global spine anatomy.
Inputs per point:
- 3D coordinates (x, y, z)
- Patient metadata (broadcast to all points)
- Gaussian noise (to increase deformation flexibility)
Feature Extraction:
- Fully Convolutional Neural Networks (F-CNNs)
- Three 1D convolution layers with output sizes: 128 → 64 → 3
Output:
- A 3D displacement vector per point
- Predicted postoperative spine = preoperative spine + displacement
Key design choices:
- Same displacement applied to all landmarks of a vertebra to preserve topology
- Noise enables differentiation between nearby points
- Metadata allows learning patient-specific and surgeon-specific correction patterns
Novel contribution: A Rigid Transformation Loss that forces the network to learn translations and rotations between consecutive vertebrae, enabling global shape awareness and preventing anatomically implausible predictions.
Evaluation Metrics
Model performance is evaluated using standard clinical indices computed from the predicted and ground-truth postoperative spines:
- Coronal Balance (°): Measures global spine alignment in the frontal plane.
- Cobb Angle – Frontal Plane (Cobb PA, °): Quantifies scoliosis severity in the coronal plane.
- Cobb Angle – Sagittal Plane (Cobb LAT, °): Quantifies curvature in the sagittal plane.
Errors are reported as Root Mean Square Error (RMSE) between predicted and real clinical measurements.
Mathematical Formulation
1-Rigid Transformation Model
A rigid transformation between two consecutive vertebrae is defined as:
\[y = Rx + t\]where:
- $x, y \in \mathbb{R}^3$ are the input and output 3D points,
- $R \in SO(3)$ is the rotation matrix,
- $t \in \mathbb{R}^3$ is the translation vector.
2-Overall Loss Function
The CPDNet training objective combines geometric accuracy and anatomical consistency:
\[L(S_{\text{Post}}^*, S_{\text{Post}}) = L_{\text{Geometric}}(S_{\text{Post}}^*, S_{\text{Post}}) + L_{\text{Rigid}}(S_{\text{Post}}^*, S_{\text{Post}})\]where:
- $S_{\text{Post}}^*$ is the predicted postoperative spine,
- $S_{\text{Post}}$ is the ground-truth postoperative spine.
3-Geometric Loss
The geometric loss measures the point-wise reconstruction error between predicted and real postoperative spine landmarks:
\[L_{\text{Geometric}}(S_{\text{Post}}^*, S_{\text{Post}}) = \sum_{p \in S_{\text{Post}}^*, q \in S_{\text{Post}}} \|p - q\|^2\]Since the model is supervised, landmark correspondences between $p$ and $q$ are known.
4-Rigid Transformation Loss
The rigid transformation loss enforces anatomical consistency by penalizing differences in inter-vertebral translations and rotations:
\[L_{\text{Rigid}} = \sum \|t - t^*\|^2 + \sum \|r - r^*\|^2\]where:
- $t, r$ are the translation and rotation vectors from the ground-truth spine,
- $t^, r^$ are those computed from the predicted spine.
Each rigid transformation is computed between consecutive vertebrae and assigned to all landmarks of that vertebra.
5-Translation Between Consecutive Vertebrae
For two consecutive vertebrae $i$ and $j$:
\[\Delta x = x_i - x_j, \quad \Delta y = y_i - y_j, \quad \Delta z = z_i - z_j\]The translation vector is:
\[t = (\Delta x, \Delta y, \Delta z)\]6-Rotation Representation (Axis–Angle)
A 3D rotation is represented using an axis–angle formulation:
\[r = n\theta\]where:
- $n$ is a unit rotation axis,
- $\theta$ is the rotation angle.
The corresponding rotation matrix is computed using Rodrigues’ formula:
\[R = \cos(\theta)I + \sin(\theta)S(n) + (1 - \cos(\theta))S^2(n)\]with:
\[S(n) = \begin{bmatrix} 0 & -n_z & n_y \\ n_z & 0 & -n_x \\ -n_y & n_x & 0 \end{bmatrix}\]7-Postoperative Spine Prediction
The final postoperative spine shape is predicted by adding learned displacements to the preoperative spine:
\[S_{\text{Post}}^* = S_{\text{Pre}} + \Delta_{\text{pre} \rightarrow \text{post}}\]where $\Delta_{\text{pre} \rightarrow \text{post}}$ is the displacement predicted by CPDNet.
Results
CPDNet is compared against:
- P2P-NET (state-of-the-art point displacement network)
- PDNet (CPDNet without rigid transformation loss)
Quantitative Results (Test Set)
CPDNet achieves the lowest error across all metrics:
- Coronal Balance: 1.5°
- Cobb LAT: 7.6°
- Cobb PA: 4.9°
These results demonstrate that:
- Adding noise improves local flexibility
- Adding metadata improves personalization
- Rigid transformation loss provides the largest performance gain
Qualitative Results
Visual comparisons show that CPDNet:
- Preserves spinal topology
- Better models thoracic and thoracolumbar corrections
- Produces anatomically plausible postoperative shapes, even with limited similar cases in training
Conclusion
This work introduces CPDNet, a supervised deep learning framework for patient-specific prediction of postoperative 3D spine shape in scoliosis surgery. By integrating:
- point-based deformation,
- clinical metadata, and
- a novel rigid transformation loss,
CPDNet successfully learns both local deformations and global spinal structure. The model outperforms existing point-based methods and demonstrates strong clinical relevance.
Clinically, CPDNet can serve as the foundation for a surgical planning and visualization tool, helping surgeons and patients better understand expected postoperative outcomes for different surgical strategies.
Resources
Please find the details of the project through the following papers:
Project repository
GitHub Code: CPDNet: Controlled Point Deformation Network for Patient-Specific Postoperative Spine Shape Prediction