How do you make it independent of input size

I understand that the brain does not care about the size of the image, to some extent. And so do the Convolutional layers because the kernel size does not need a specific input.

However we define an input size for the neural models, and then it happens that we can not input an image of a different size, but this should not actually matter ?

So the way we do it is getting rid of the input layer, just like that, or there is any way to just input an image of any size (even if less accurate) ?