SIAM Undergraduate Research Online
Volume 17
In This Volume
-
Evaluating Activation Functions with nAI and Quadrature Based Neural Networks
Published electronically December 6, 2024DOI: 10.1137/24S1658528
Authors
John C. Breedis (Corresponding author – University of Texas at Austin)
Project Advisors
Dr. Tan Bui-Thanh (University of Texas at Austin)
Abstract
We provide a new perspective on comparing activation functions via neural network approximate identity (nAI), a family of bell-curve activation functions Bθ with parameter θ > 0. This was done by comparing the performance of several neural networks across multiple data sets, where each neural network has one hidden layer and uses nAI as activation functions. The data sets used include polynomial, discontinuous, and non-differentiable regression problems, as well as MNIST classification. Across all regression problems, hinge activation functions outperformed sigmoidal activation functions. For classification, sigmoidal activation functions perform slightly better than hinge functions. In addition to traditional neural networks, we develop a machine learning architecture that utilizes integral convolution and quadrature by fixing the initial weight and bias matrices of the network. Supplemental modifications were implemented to improve the accuracy of these networks. This modified architecture was used alongside standard Feedforward networks across each regression problem with similar efficacy.
-
Accurately Classifying Out-Of-Distribution Data in Facial Recognition
Published electronically November 13, 2024 -
Optimizing Energy Functional in Wave and Heat Equations with Initial Conditions in a Class of Rearrangements
Published electronically October 30, 2024 -
Travelling Waves of the Diffusive Streeter-Phelps Equations with Braun-Berthouex BOD Decay
Published electronically October 24, 2024 -
Machine Learning for Hotel Reservation Prediction
Published electronically October 17, 2024 -
Mixed Precision MINRES
Published electronically October 4, 2024 -
Predicting Market Share: Application of Alternative Lotka-Volterra Competition Model
Published electronically September 23, 2024 -
Mean Imputation and Stochastic Coordinate Descent for Linear Systems with Missing Data
Published electronically August 29, 2024 -
MCRAGE: Synthetic Healthcare Data for Fairness
Published electronically August 13, 2024 -
Statistical Methods Applied to Gene Expression Data to Explore Cancer Features
Published electronically June 27, 2024 -
Using a Smartphone Accelerometer to Classify Longboarding Motion
Published electronically June 20, 2024 -
Fast & Fair: Efficient Second-Order Robust Optimization for Fairness in Machine Learning
Published electronically June 11, 2024 -
Comparative Evaluation and Refinement of Linear Algebra-Based Camera Calibration Algorithms
Published electronically June 7, 2024 -
Pooling Matrix Designs for Group Testing
Published electronically May 30, 2024 -
Predictive Modeling of H5N1 Bird Flu in United States of America: A 2022-2023 Analysis
Published electronically May 17, 2024 -
Quantification of the Effects of Voter Protocols on the Outcome of Approval Voting
Published electronically April 16, 2024 -
Modeling Renewable Electricity Purchasing for Sustainable Management of Clarkson University's Energy Portfolio
Published electronically April 9, 2024 -
An Infectious Disease Model with Asymptomatic Transmission and Waning Immunity
Published electronically March 29, 2024 -
Predator-Prey Oscillations in a Cellular Automaton of Huffaker's Mite Experiment
Published electronically February 13, 2024 -
How to be #1 in the IOI? A Study on Rating Nations Participating in the International Informatics Olympiad
Published electronically February 9, 2024 -
From Pole to Podium: Adjusting Elo Method to Separate Car and Driver in Formula One Racing
Published electronically February 6, 2024 -
Identifying Priority Areas for Expanding Mental Health Facilities with Mixed Integer Linear Programming
Published electronically January 25, 2024
Become a SIURO Author
Publish your undergraduate research with SIAM to experience all aspects of the peer review process — from submission, review and revision, to publication.
Submit a Paper to SIUROStay Up-to-Date with Email Alerts
Sign up for our monthly newsletter and emails about other topics of your choosing.