14.04 - How to disable ssh login for all users except one?

I Have ubuntu server 16.04, running on localhost. I want to disable ssh login for all users except john. I have tried all answers and comments posted here, nothing works. For example

Match User !john

PasswordAuthentication no

I put it at the end of /etc/ssh/ssh_config, no empty lines after it, and I restart the server sudo service ssh restart. Nothing works.

I think you can use the following directive and do not need Match section:

AllowUsers john