Challenges and Solutions
An analysis of the technical obstacles encountered during the development of HERMES OPTIMUS and the methodological approaches implemented to resolve these challenges.
Memory Constraints
The TI-84 Plus Silver Edition operates with a mere 24KB of RAM, presenting an exceptionally constrained environment for neural network implementation. The initial architectural approach attempted to incorporate all weights and biases directly within the program code structure, which inevitably resulted in memory overflow errors due to the substantial parameter requirements.
Challenge
A 4-60-12 neural network requires storing 240 weights for the first layer, 720 weights for the second layer, and 72 bias values—a total of 1,032 floating-point values.
Solution
Instead of hardcoding the weights in the program, I converted them to external .8xm (matrix) and .8xl (list) files that could be loaded at runtime. This reduced the program size by approximately 80%.
Training Limitations
The initial research methodology involved training the neural network directly on the calculator hardware utilizing standard backpropagation algorithms. This approach encountered multiple technical limitations, including prohibitively slow execution performance and critical variable overflow errors that prevented successful training convergence.
Challenge
Training on the calculator was impractically slow, taking approximately an hour for 25 epochs. Additionally, the backpropagation algorithm caused variable overflow errors due to the limited precision of TI-BASIC's floating-point implementation.
Solution
I developed a Python script to train the network externally and then transfer the trained weights to the calculator. The external Python training required 500,000 epochs for the 12-word dictionary, compared to 50,000 epochs for the earlier 4-word version. Based on the calculator's performance of approximately 25 epochs per hour, training the full 12-word dictionary would have taken over 20,000 hours—or more than 2 years of continuous operation. The calculator's batteries would have expired approximately 400 times before training completion, assuming they last about 50 hours of heavy computational use. This comical realization further justified the decision to train externally, as waiting multiple years for a calculator to train a neural network would certainly test the limits of both the hardware and the researcher's patience.
Weight Transfer Process
Following the successful training of the network using Python, it became necessary to develop an efficient methodology for transferring the optimized weights to the calculator hardware. The preliminary approach involving manual transcription of weights through TI-BASIC commands proved both inefficient and susceptible to transcription errors.
Challenge
The original plan was to generate TI-BASIC code with commands like "0.123→[I](1,1)" for each weight, but this approach was impractical for over 1,000 weights and resulted in a program that was too large to run on the calculator.
Solution
I discovered that I could convert the weights to CSV files and then use Cemetech tools to convert these CSV files to .8xm and .8xl files that could be directly loaded onto the calculator. This eliminated the need for TI-BASIC commands to initialize the weights.
Numerical Stability
The sigmoid activation function, while mathematically appropriate for the classification task, can generate numerical values at the extremes of the computational range, potentially resulting in overflow or underflow errors when executed on calculator hardware with limited numerical precision.
Challenge
When calculating sigmoid(x) = 1/(1+e^(-x)), large negative or positive values of x would cause errors on the calculator due to its limited numerical range.
Solution
I implemented safeguards that clip extreme values before applying the sigmoid function:
If L₂(I)>10:Then:0.9999→L₂(I):End
If L₂(I)<10:Then:0.0001→L₂(I):End
If abs(L₂(I))≤10:Then:1/(1+^(L₂(I)))→L₂(I):End
Performance Optimization
The expansion of the system's dictionary size from the Legacy version of HERMES introduced significant performance challenges. The original implementation with only four words (each with four letters) demonstrated substantially faster processing times.
Challenge
The increased dictionary size in the current version of HERMES OPTIMUS resulted in longer processing times, increasing from approximately 7 seconds with the smaller dictionary to about 20 seconds with the expanded vocabulary. This performance degradation significantly impacted the system's practical utility.
Hardware Limitation
This performance constraint is fundamentally tied to the Zilog Z80 processor's limited 15 MHz frequency. Despite implementing various code optimizations such as minimizing variable reassignments, reducing conditional checks, and using more efficient loop structures, the processing speed remains inherently restricted by the calculator's hardware architecture. The trade-off between dictionary size and processing speed represents an unavoidable limitation of implementing neural networks on such constrained hardware, highlighting the significant achievement of getting the system to function at all on such limited computational resources.
Update: Code Optimization
May 2025 Update: Reddit user u/TheFinalMillenial has optimized the HERMES OPTIMUS code, achieving significant performance improvements:
- 25% faster execution speed
- 30% smaller code size
- Reduced processing time from ~20 seconds to ~13 seconds on the TI-84 Plus Silver Edition
- Improved autocorrection accuracy to approximately 95%
Key Optimization Techniques:
- Simplified if statements by omitting Then and End statements when only one line of code is executed
Before: If A=0:Then:Disp A:End
After: If A=0:Disp A
- Omitted trailing close parentheses in TI-Basic to improve performance in For loops
Before: L1(A)->B
After: L1(A->B
- Utilized the Ans variable which is extremely fast and contains the value of the last calculation
Before: 1+A->A:A->B
After: 1+A:Ans->B
- Avoided using Goto jumps inside if statements or loops to prevent memory leaks
Before: If A=0:Then:Goto B:End
After: If A=0:Goto B
Additionally, the optimized code fixed a bug in the confidence level percentage calculation, further improving the accuracy of the neural network's predictions.
The full optimized code and detailed explanation can be found in u/TheFinalMillenial's Reddit comment and on their GitHub as well as mine.
Final Solution
The culminating implementation of HERMES OPTIMUS successfully addresses all identified technical challenges, resulting in a fully operational neural network system that functions effectively within the constraints of the TI-84 Plus Silver Edition calculator hardware.
The critical technical innovations that facilitated this achievement include:
- External training in Python with a compatible architecture
- Converting weights to .8xm and .8xl files for efficient loading
- Implementing numerical safeguards to prevent overflow errors
- Optimizing the forward pass for speed and memory efficiency
- Designing a user-friendly interface within the calculator's constraints
These methodological solutions collectively reduced the program's memory footprint by approximately 80%, thereby enabling the successful execution of a neural network on a computational device with merely 24KB of RAM utilizing a programming language developed in the 1980s—a significant achievement in resource-constrained machine learning implementation.