Data Pre-processing Modular

Sample Entropy (SEn)

Sample entropy (SEn) is a powerful statistical measure used to assess the complexity of time series data and to quantify the likelihood that a new pattern will emerge. One of its key advantages lies in its insensitivity to the length of the data set, making it particularly valuable in scenarios where the size of the dataset varies significantly. This attribute allows researchers and data scientists to apply SEn reliably across different temporal readings without concern for biases introduced by differing data lengths (Manis et al., 2024).

Mathematical Description of Sample Entropy

To understand how SEn works, let’s consider a time series denoted as (\mathbf{X}(L) = [x(1), x(2), \cdots, x(L)]) where (L) is the length of the series. For a given dimension (m), we can define a vector of (m) consecutive values as follows:

[
Y_{m}(i) = [x(i), x(i+1), \cdots, x(i+m-1)]
]

where ( 1 \leq i \leq L – m + 1 ). The distance between two vectors (Y{m}(i)) and (Y{m}(j)) is given by:

[
d(Y{m}(i), Y{m}(j)) = \max_{k=0,1,2,\cdots,m-1} \left{ |x(i+k) – x(j+k)| \right}
]

Next, let (B{i}) denote the number of instances (j) such that (d(Y{m}(i), Y{m}(j)) \leq r) and (A{i}) denote the number of instances (j) such that (d(Y{m+1}(i), Y{m+1}(j)) \leq r). The sample entropy can then be defined as:

[
SEn(m, r, L) = -\ln \left[ \frac{A^{m}(r)}{B^{m}(r)} \right]
]

where (B^{m}(r) = \frac{B{i}}{N – m – 1}) and (A^{m}(r) = \frac{A{i}}{N – m – 1}).

Variational Mode Decomposition (VMD)

VMD is an innovative technique that facilitates effective pre-processing of data by decomposing non-stationary time series, such as daily evaporation readings, into well-defined frequency bands, known as intrinsic mode functions (IMFs).

Mathematical Description of VMD

The model for decomposing a signal (f(t)) into its IMFs is mathematically represented as:

[
y{k}(t) = a{k}(t) \cos(\varphi_{k}(t))
]

where (a{k}(t)) indicates the instantaneous amplitude and (\varphi{k}(t)) is the phase. The VMD process is formulated as a constrained variational problem whereby:

[
\mathop{\min}\limits{{y{k}},{\omega{k}}} \sum\limits{k=1}^{K} | \partial_t\left[ (\gamma(t) + \frac{j}{\pi t}) y_k(t) \right] e^{-j\omegak t} |{2}^{2}
]

This mathematical framework allows for efficient extraction of essential features from the data based on the decomposed IMFs.

Meta-Heuristic Algorithms

Whale Optimization Algorithm (WOA)

The WOA is a powerful meta-heuristic algorithm modeled after the hunting strategy of humpback whales. It is particularly adept at solving nonlinear optimization issues and features mechanisms like a local search through a “shrink enveloping mechanism” and a global search via a “random search strategy.” Its strong convergence and precision make WOA suitable for a variety of optimization problems (Mirjalili & Lewis, 2016).

Mathematical Overview of WOA

In WOA, the position of the humpback whales during optimization is updated based on the following formulas:

1. For (p < 0.5):
   [
   Y(t + 1) = Y^{}(t) – A \left| CY^{}(t) – Y(t) \right|
   ]
2. For (p \geq 0.5):
   [
   Y(t + 1) = \left| CY^{}(t) – Y(t) \right| e^{bt} \cos(2\pi l) + Y^{}(t)
   ]

In these formulas, (Y^{*}(t)) and (Y_{rand}(t)) refer to the best and randomly selected whale positions, while (A) and (C) dictate the hunting strategies employed.

Sparrow Search Algorithm (SSA)

SSA is a newer, efficient swarm intelligence algorithm inspired by the foraging and anti-predation behaviors of sparrows. It exhibits improved optimization abilities and faster convergence rates compared to other algorithms like WOA and Particle Swarm Optimization (PSO) (Xue & Shen, 2020).

Mathematical Representation of SSA

In SSA, the position of the producer at iteration (t+1) can be represented as:

1. If (R{2} < s):
   [
   x{ij}^{t + 1} = x{ij}^{t} \exp(-i/\alpha \cdot it{\max})
   ]
2. If (R{2} \geq s):
   [
   x{ij}^{t + 1} = x_{ij}^{t} + Q \cdot L
   ]

Here, (s) represents a safety threshold and (R_{2}) signifies an alarm value. The algorithm dynamically adjusts according to the varying threat conditions observed in the environment.

Deep Learning Models

Convolutional Neural Network (CNN)

CNNs are designed to extract spatial features from input data through a hierarchy of convolutional and pooling layers. These networks perform well at local feature extraction, which is crucial for handling intricate datasets like time-series data.

Bidirectional Long Short-Term Memory Network (BiLSTM)

BiLSTM is an advanced recurrent neural network (RNN) design dedicated to preserving context in sequential data by processing information both forwards and backwards. This dual-phase examination allows BiLSTM to more effectively manage long-term dependencies compared to its predecessors like GRU and standard LSTM (Song et al., 2024).

Hybrid Forecasting Models

In the current investigation, an integrated hybrid model combining WOA-VMD and SSA-CNN-BiLSTM has been devised. WOA is employed for optimizing the parameters of VMD, while SSA fine-tunes the hyperparameters of the CNN-BiLSTM model.

• Step 1: Define the number of whales and initialize their positions representing parameter pairs ( (K, a) ).
• Step 2: Decompose the time series for each position.
• Step 3: Compute SEn for the decomposed components, using it to evaluate model performance.
• Step 4: Update whale positions, iterating through the aforementioned steps until convergence.

Parameter Settings and Model Evaluation

The WOA was employed to optimize parameters (K) and (a) within ranges defined for effective performance. A structured approach including cross-validation was taken to thoroughly assess model capabilities, while statistical metrics such as RMSE and R² were employed for performance evaluation.

Through this structured integration of various methods, the model aims to refine predictions about time series trends, particularly important in fields such as agriculture, climate studies, and resource management. The advanced modeling strategies employed promise enhanced accuracy and understanding in forecasting applications.