Supplementary Appendix

TacEvo: Architectural Variants, Prompts & Validation

Appendix A. Architectural Modifications in Force Variants

Variant 1

• ReLU → LeakyReLU
• Extra Conv layer in every block
• 3 → 2 Conv Blocks
• Spatial Attention Block
• Less Neurons in FCN hidden layer 512 → 256
• MaxPool → strided conv

Variant 2

• Extra Conv layer in every block
• 3 → 2 Conv Blocks
• Spatial Attention Block
• Less Neurons in FCN hidden layer 512 → 256
• MaxPool → strided conv

Variant 3

• 3rd Conv block does three 1x1 conv instead of 3x3 conv
• Spatial Attention Block
• MaxPool → strided conv

Variant 4

• 4th Conv Block
• Spatial Attention block
• MaxPool → strided conv

Appendix B. Architectural Modifications in Grating Variants

Variant 1

• ReLU → GELU
• Wider conv layers
• 2 FCN → 4 FCN
• Removed Dropout
• Added batchnorm and layernorm after FCN

Variant 2

• Wider conv layers
• 2 FCN → 4 FCN
• Removed Dropout
• Added batchnorm and layernorm after FCN

Variant 3

• Wider conv layers
• ReLU → GELU
• Residual Blocks (skip-connections)
• 2 FCN → 4 FCN
• Removed Dropout
• Added batchnorm and layernorm after FCN

Variant 4

• Wider conv layers
• Residual Blocks (skip-connections)
• 2 FCN → 5 FCN
• Removed Dropout
• Added batchnorm and layernorm after FCN

Appendix C. List of Prompts Used for Force and Grating Tasks

Force Regression Task Prompts

General Refinement

• Refine the model to minimize Mean Squared Error (MSE) for a regression task, without altering the I/O layers.
• Re-balance the architecture for a better trade-off between parameter count and prediction accuracy.

Width and Channel Adjustments

• Widen the convolutional base by increasing the output channels of all `nn.Conv2d` layers by 25%.
• Make the regressor head more powerful by increasing its hidden layer neurons from 512 to 1024.
• Decrease the networks width by halving the output channels of the `conv1` and `conv2` blocks.

Component & Parameter Tuning

• Replace the `nn.ReLU` activation functions with `nn.LeakyReLU(negative_slope=0.01)` to handle negative values.
• Remove the `nn.Dropout` layer from the regressor head, as it can sometimes harm regression performance.
• Change the kernel size in the `conv1` layer to 7x7 for a larger initial receptive field, adjusting padding accordingly.
• Replace the `nn.MaxPool2d` layers with a strided convolution with a kernel size of 2 and stride of 2.

Layer Addition (Normalization & Regularization)

• Insert a `nn.BatchNorm2d` layer within each convolutional block to improve gradient flow.
• Add a `nn.LayerNorm` layer to the regressor head for stable activation statistics.
• Integrate a simple spatial attention mechanism before the `adaptive_pool` layer.
• Incorporate an additional `nn.Linear` layer with 256 neurons into the regressor head.

Structural & Final Layer Changes

• Introduce a single residual connection that adds the output of `conv1` to the output of `conv3`. Ensure the residual connection has matching spatial dimensions and channels.
• Replace the standard convolution in the `conv3` block with a depthwise separable convolution to make the model lighter.
• Add a `nn.Sigmoid` activation to the final output layer to ensure all predictions are between 0 and 1.
• Add a `nn.Tanh` activation to the final output layer to constrain predictions between −1 and 1.
• Add a `nn.Hardtanh` layer to the final output to clip predictions within a specific min-max range.
• Refactor the regressor head into its own `nn.Module` class for better encapsulation. Keep the main ImageRegressionCNN class as the primary network class.
• Make the model slightly deeper by adding a small convolutional block with 1x1 convolutions after `conv3`.

Grating Classification Task Prompts

General Refinement

• Refine the model to achieve higher classification accuracy while preserving the I/O structure.
• Optimize the model for faster inference speed by making minor architectural adjustments.

Width and Channel Adjustments

• Widen the network by increasing the output channels of the `conv2` block by 25%.
• Make the network slightly narrower by decreasing the number of neurons in the hidden fully connected layer to 384.
• Increase the complexity of the classifier head by adding a second hidden layer with 256 neurons.

Component & Parameter Tuning

• Replace the `nn.ReLU` activation functions with `nn.GELU` for potentially smoother gradients.
• Adjust the dropout probability in the classifier head to 0.6 for stronger regularization.
• Change the kernel size in the `conv3` block to 5x5, adjusting padding to maintain the feature map size.
• Replace the `nn.MaxPool2d` layers with `nn.AvgPool2d` for a different pooling strategy.

Layer Addition (Normalization & Regularization)

• Insert a `nn.BatchNorm2d` layer after each `nn.Conv2d` layer to stabilize training. Use LayerNorm instead of BatchNorm in the classifier to avoid batch size issues.
• Add a `nn.LayerNorm` layer to the classifier head just before the `nn.ReLU` activation.
• Incorporate an additional `nn.Dropout` layer after the first `nn.Linear` layer in the classifier.

Structural & Final Layer Changes

• Introduce a single residual connection that skips over the `conv2` block.
• Change the final activation to a standard `nn.Softmax(dim=1)` for direct probability outputs.
• Replace `nn.AdaptiveAvgPool2d` with a combination of adaptive average and max pooling to capture more diverse features.
• Refactor the three `nn.Sequential` convolutional blocks into a single custom `nn.Module` for better code organization.
• Make the classifier head more lightweight by reducing its hidden layer size and removing dropout.

Appendix D. System Prompts

Force Regression System Prompt

The following system prompt is used to instruct the LLM on how to modify the baseline architecture for the Force Prediction (regression) task.

Grating Classification System Prompt

The following system prompt is used to instruct the LLM on how to modify the baseline architecture for the Grating Classification task.

Appendix E. Baseline Model Validation on Hardness Prediction Task

The focus of the development of TacEvo in the scope of this project is the application of the pipeline to the Force Regression and Grating Classification tasks. However, these two tasks are relatively easier to be learned when compared to more complex tasks such as Hardness Regression. The main bottleneck in TacEvo's performance on a different task, such as hardness, would be in the robustness of the base architecture (Figure 2) at learning that particular task. After validating the baseline's performance, the TacEvo pipeline performance is not expected to differ significantly from the conducted studies, since generating and evaluating variants would only minimally differ in performance from the baseline performance. Therefore, this is a supplementary evaluation and validation of the model's robustness on the hardness dataset sourced from ViTacTip.

The performance of the baseline model on this Hardness Task is presented without any further involvement in NAS. The baseline model was trained with the same parameters as those presented in Section V-A. The final layer was modified to output a single continuous value corresponding to the hardness of the object in Shore units. In Table III the reported R² value is 0.990, which means that the model was able to explain 99% of the variance present in the data, indicating that the model has fitted well around the dependent variable (predicted hardness values). Also, since the hardness value labels of the dataset ranged between 17.5 and 30.0 Shores, the model provided test error values that are considerably small compared to the prediction range. The Root Mean Squared Error value reports the error in the same units as the prediction, showing a 0.430 error, indicating that the baseline architecture is capable of generalising to tasks and datasets beyond the two analysed in this study.