Sometimes a visitor will arrive at your webserver by specifying the target (your server) by IP address only. This means they will get the first default vhost. It also means they will get an SSL error (mismatching name). This article shows how you can block that with a 403.
Put the following into your vhosts. If you know your primary (default) vhost, you can simply place it there. Or in a “.htaccess” file.
Replace the IP address “10.1.2.3” with your servers actual public IP address.
RewriteEngine On RewriteCond %{HTTP_HOST} 10.1.2.3 RewriteRule .* - [F]
Reference: https://serverfault.com/questions/511413/in-apache-how-do-i-disable-access-to-a-website-when-someone-only-uses-the-ip-ad#511451