Best optmization algorithm

Hello Community
I have just developed an algorithm (I can say a family of algorithms) which have as performance a fast convergence while maintaining the generalization
I ask you for any help and advice
I want to join a company as a researcher in the field of artificial intelligence. having access to very expensive hardware (like GPUs) and more…
Here are the results of one of my algorithms in MNIST and IMBD data.( conduct on personal computer)

our1
our2
our3
our4
ours1
ours2
ours3
ours4

sincerely

Hi there,

This seems interesting. I do observe, however, that certainly for a low number of epochs, your algorithm has lower accuracy compared to the industry standards. Perhaps your approach has some other benefits that are not represented on these graphs? (time wise?)

Cheers,
Timo

Hi there
As we know, there are two types of complexity (timing and memory requirements)
In this algorithm, I managed to achieve both qualities: less memory needed compared to adam and Adagrad RMSProp and high accuracy in finite time, even though I trained my algorithm with some time-consuming epochs on a personal computer.
I also reduced the hyperparameter used in many algorithms.

do you suggest any further research in this field?
cheers

I suggest to take a look at some strong benchmark protocols like ICLR 2021:

2 Likes

Thank you sir, this is very helpful

sincerely :innocent:

1 Like

Awesome work. There are a few other notable optimizers that may be worth comparing your work to, although I’m no expert:

Thank you sir for your suggestions :innocent :innocent: