@words Unless you have a fixed IP address (Public) that is reserved for the device you use to connect to the VPN you will not be able to block port scanners and only your encryption and authentication would be your line of defense.  However, if you use a operator/ISP with known IP ranges you can lookup the ranges and use the link shared in the other post to allow only IPs coming from those ranges. You may need to keep watching which IPs get assigned to figure out the range your service provider assigns to the device you use to connect to the VPN.  Usually ISPs/Mobile operators have dedicated ranges for mobile, residential and corporate. You then use such range with cidr such as 10.105.32.0/19 .

  @yabdali ok I thought there was a way to add a "blacklist" of IPs, whom inbound connections/packets would just get dropped...

This is not possible??

  @words Yes you can, you just need to add each IP address to a group and block that group. If you follow the article in the link posted above you can see that you have to go to Preferences > IP Group > IP Address (Tab) and add each ip (e.g 169.228.66.212 /32) and give it a name then add the other IP with /32 and give it a name. After that, you to other (Tab) IP Group and Add an entry in the Group List, fill the Group Name and pull the Address Name drop-down menu, tick those IP names you created previously and click OK.

Then go to Firewall > Access Control and click Add. Enter a name, Policy (Block) , Service Type All, Source: Your IP Group Name. and click OK. 

Keep in mind that with this approach, everytime you get an IP scanner hitting your IPSec port you will have to add it to IP Group.

  @yabdali that allows one to enter a whole network, not a group where I can add different IPs.

See below what I'm seeing in the log file, these are not VPN clients of mine nor am I trying to connect any kind of site 2 site tunnel with any of these. So what are they??

So looks like there is some fishing going on from these IPs, the log files don't really give any more info, not sure if I need to turn on DEBUG or something here....

  @words You will need to choose IP Address / Mask, enter an IP such as 64.62.197.66 with subnet 32, repeat the same for each IP.  As I said, you will have to add every IP you that hits your firewall.  Thats why I said you are better off with only allowing your ISP IP addresses range and allow only traffic from ISP IPs group so you don't have to worry about adding separate IPs.  I assume you are starting to learn about networking so you may need to read more and get familiar with the basic concepts of frewalls and networking.