Fortigate Issue with VLAN's and Routing

[giantsnyy]

Hi Everyone,

 

I'm having a bit of an issue with my FortiWifi 30D and VLANs.

 

So... I've tried everything.  I need to create 4 separate VLANS (which I know this Fortigate can handle), all of which, except 1, route out to the internet.  I've been able to create the VLAN's, create the necessary firewall policies which allow external communication, create the firewall policies which allow intra-vlan communication (where necessary) and test intra-vlan communication.  Everything works, except outside communication.  Every time I try to ping out using VLAN's other than VLAN1, i get network unreachable.  VLAN1 works without an issue.

 

I have even tried separating out the physical interfaces instead of using VLAN's - and the result is the same every time.  Every connection works, except outbound to the internet on the separate VLAN's.  VLAN1 is the only vlan to access the internet.  I've read places that I need to create a static route for the outbound, but this customer doesn't have a static IP address.  They only have DHCP from Verizon FiOS.  This is a residential customer of mine, and not a business - so they can't get FiOS business for a static IP.

 

Here's the specs:

Verizon FiOS Gigabit (No Static IP)

FortiWifi 30D

Ubiquiti UniFi Access Points (3)

vlan1 - Wired Network and Wireless Laptops | Firewall rule internal > wan1 all/all allowed, wan1 > internal all/all denied, internal > vlan60-dvr all/all allowed, internal > vlan33 all/all allowed (THESE WORK)

vlan33 - Phones, iPads, Fire TV Sticks/Etc. | Firewall rule vlan33 > wan1 all/all allowed (DOES NOT WORK), vlan33 > internal all/all allowed (Works)

vlan50 - Guest Wireless | Firewall rule vlan50 > wan1 all/all allowed (DOES NOT WORK)

vlan60 - CCTV Cameras (vlan1 accesses DVR only - no internet access required) | Firewall rule vlan60 > wan1 all/all allowed (Does not work) vlan60-dvr > internal all/all allowed (works)

 

I gave up on the VLAN's and tried physical interfaces with the same rules and the same results applied:

 

internal(lan/port1) - vlan1

lan2 (port2) -vlan33

lan3 (port3) - vlan50

lan4 (port4) - vlan60

 

Thanks,

Mike

 

Edited January 2, 2018 by giantsnyy

[farmeunit]

Were doing our VLANs on our core switch, but I do know that I had to create a static route manually.  It says it will create one automatically, but it did not for me.

 

We only have two.

 

172.16.0.0/12 with gateway of 192.168.1.1 which is our incoming (port 10)

0.0.0.0/0 with a gateway of 64.237.17.49 which is our outgoing (port 9)

 

I assume you'll need a static route for each VLAN, incoming and outgoing.

 

I'm not an expert, by any means, but we have a 600D here.  On a side note, I read something about ports 1 and 2 being reserved for something.  It was in release notes for an upgrade, so you might do some research on it if you have or will be upgrading soon.

 

 

[sc302 Veteran]

I am not familiar with fortigate, but it sounds as if you don't have your ACL's set right or you screwed with the default rules/routes that were setup.  Don't screw with the routes, screw with the ACL's.  You would have to create an allow all but deny your internal traffic. 

 

 

 

[giantsnyy]

I haven't touched the routes - the default route is automatically set since the wan port is set as DHCP.

 

I've tried all combinations of ACL's.  I managed to get one VLAN working correctly (aside from the management vlan).  I've mimicked the ACL on the other vlans... but they still won't work.

[sc302 Veteran]

I hate saying it but sounds like a call to support.

[giantsnyy]

I'd love to - but this customer purchased the fortigate 2+ years ago, and hasn't renewed a support contract due to cost.

[sc302 Veteran]

If you want, I can take a look at it.  I make no promises or warranty my work. 

 

It is probably $100 to get that thing under warranty....well worth it imo....or they can continue to pay you to screw with it which will cost them more in the long run than to get support and be done in 15 minutes.  The longer you screw with it unsuccessfully the more you get paid to not accomplish your job, the more it will cost them.  If you have spent more than 3 hours on it, you have spent more than a support contract would cost them.

[+BudMan MVC]

11 hours ago, sc302 said:

It is probably $100 to get that thing under warranty

Its a bit more than that But if they do not renew then they don't get any of the bells and whistle like the av scanning website filtering, etc. etc. Full bundle for a year is prob 200.. I think a support package is like $60 without any of the bells and whistle other features.

 

How much is your time going to cost - I would assume way more than that 200..

 

So can your vlans/networks talk to each other?  If so then you prob need an outbound nat rule.. This would be the case with pfsense - but pfsense does it automatic.. Unless the user tuns off automatic outbound nat - which happens a bunch

 

How exactly do you have this all connected when you broke out your networks/vlans - are they wireless?  But sounds like your vlan/network traffic works - its routing through the firewall?  If so then yeah I would bet its your outbound nat rule.  It doesn't know how to nat the new networks to is public IP... This is done in pfsense here.. I doubt it looks like this in the fortinet thing - but just to show you what I am talking about with the outbound natting.

 

 

So you can see the automatic rules take all my local networks and can nat them to the WAN interface IP.. to get to the internet.  I also have a manual rule that allows natting to a vpn client interface IP, etc. Get the idea?  That would be my bet to the problem if your internet network traffic is working.

[sc302 Veteran]

It is pretty low end....I think buying a new one was around 3-400.  I don't know if that includes other services, but they weren't that expensive.  He wouldn't need to get the other services to have them setup the vlans and access rules.  

 

We could probably configure it if we had the device in front of us.  Not something I am going to take a stab at without having it or having access to it. 

[+BudMan MVC]

yeah shouldn't be hard to figure out.. Just need access to it.. If his intervlan traffic is working but internet is not I would bet large sum its outbound nat not setup.

 

Agree I believe 1 year support without any bells and whistles... here you go $52

https://www.corporatearmor.com/product_info.php?products_id=7412

Fortinet FortiGate-30D / FG-30D Support 8x5 FortiCare Contract 1 Year (New Units and Renewals)

[grunger106]

If you ping out from a VLAN other than one what is the exact response and where is it from?
 

My suspicion is that you do need a static route set for each VLAN, (nothing to do with static addresses though) you need to it where to send packets that it doesn't know what to do with itself, this will likely be done for you for VLAN1

So each requires a route set to 0.0.0.0 with the next hop being WAN1

 

If this was an SNAT issue wouldn't we be seeing requests timing out, rather than Destination Net Unreachable?

 

But I know nothing about Fortinet......

[+BudMan MVC]

9 hours ago, grunger106 said:

My suspicion is that you do need a static route set for each VLAN

Sorry but no, there is no router or firewall that ever needs a static route for anything that is directly attached to it.  No devices needs to have anything added as route for a network it is directly attached too.

 

If device has an interface in a network, with an address on it - then it automatically knows how to talk to that network..  You need to "add" a route to something to tell it how to get to some other network that it is not attached to via a network it is attached too..

 

So you have a router that interfaces in 192.168.1/24 and 10.0.0/24 and 172.16.0/24

 

And it needs to know how to get to network 4.5.6.0/24, you could add route that says hey to get to 4.5.6.0/24 you talk to 172.168.0.254, you might add route that says to get to 7.8.9.0/24 talk to 192.168.1.254, etc..   You do not need to tell the router how to talk to 192.168.1.x or 10.0.0.x or 172.168.0.x since it is directly attached to those networks.

 

Look at the routing table of your computer.. Did you have to add a route for the network its attached too?

[giantsnyy]

Sorry for the delayed replies.

 

It ended up being the switch that was in place.  Something was messed up with one of the SFP’s I was using in the LACP trunk to the second floor switch.  I swapped it and the VLANS started working correctly.

 

As for why the Fortigate itself couldn’t ping out from those interfaces, I’ll never know.  There are so many little bugs with the low end fortigate firmwares that I’ve decided to stop using them altogether.  For example - when establishing a site to site IPSec tunnel, you can’t ping to the remote subnet and traceroutes die off.  This must just be another one of those bugs.

 

We weren’t happy with the throughput (clients wouldn’t get above 150Mbit connected to it) from the fortigate either so it was swapped with a UniFi Security gateway pro. 

 

 

[farmeunit]

I have a friend that loves the Unifi Security gateways.  I haven't used, but like the rest of their stuff.  It's probably fine for a lot of small businesses, especially if you don't need all the extra stuff.  

 

IPS and AV is pretty handy, though.  I did notice they have IPS in beta form.

https://community.ubnt.com/t5/UniFi-Stories/New-Beta-UniFi-Intrusion-Prevention-System-stops-hack-attempt/cns-p/2169344

 

 

[+BudMan MVC]

I ran a usg 3p for a bit - for 100$ it was a slick little device.. But its not even close to what pfsense can do..

 

I have it in a box if you want to buy it?  Giving out a neowin discount   Now that my pfsense up and running on again on hardware vs vm.

 

I needed something quick that could handle the new 500/50 internet connection - since me pfsense running in a VM on really old HP N40L microserver could only handle about 120mbps...  But to be fair - if you turned off the hardware offloading on the usg.. It could only do about the same.. Its the hardware offloading that allows it to do the 500 down..

 

Give them a bit and they will be nice for sure - huge fan of their APs though..

[aus73]

I know I'm a bit late to the party, but I just stumbled on to this thread with a similar issue. I actually found the answer you were looking for as I was reading through this thread. (I'm a Cisco guy moving in to the FortiGate world.) FortiGates and FortiLink handle the VLAN communication differently than most other devices. The FortiGate views each VLAN as a separate interface, an SVI basically; so, in that regard it's not different than other networking equipment. But, the FortiGate is an NGFW; so, it's not *just* a firewall or *just* a router or *just* a web filter etc. It's all those in one device. And by default, ICMP packets are not passed between interfaces in an NGFW without some intervening technology for security reasons, at least that's the way the FortiGate works. For FortiGates, that intervening tech is called a Zone. If you create a Zone and add all the VLANs to it, the Zone then becomes the interface. I'm currently working in a test bed to see what the limitations of Zones are and how I can use them. Otherwise, I'd give you more info on how they work and what their limitations are. Here's the recipe from the FortiNet Cookbook: https://cookbook.fortinet.com/using-zones-to-simplify-firewall-policies-56/

[+BudMan MVC]

5 minutes ago, aus73 said:

by default, ICMP packets are not passed between interfaces in an NGFW without some intervening technology for security reasons,

That is not anything new to NGFWs - that is just standard firewall from the days of the first packet filters..

[aus73]

On 1/4/2018 at 1:50 AM, BudMan said:

Sorry but no, there is no router or firewall that ever needs a static route for anything that is directly attached to it.  No devices needs to have anything added as route for a network it is directly attached too.

Not true. The need for routing is determined by subnets. While the router or firewall does know about the networks connected directly to it, if the logical networks are different (i.e. - the subnets are different) the device does not forward the traffic between them UNLESS you tell the device to do that. In traditional L3 devices, that means a route has to be established either via a static route or one of the routing protocols (RIP/RIPv2/RIPng, OSPF, EIGRP). In an NGFW, the security policy does the routing. And in 1st and 2nd gen firewalls, they are really just packet filters; so, they don't do any forwarding whatsoever. The 1st gens just looked at source, destination, and port, and it did what the first ACE that matched the traffic told it to do, either permit or deny, in a top down search. If it didn't find a match, the packet was dropped by default. 2nd gens did the same thing as well as look at the state of the packet (established or not), and was able to make decisions on whether to drop or permit based on that. In both cases, if the packet was permitted, it still needed a separate device to route.

[aus73]

8 minutes ago, BudMan said:

That is not anything new to NGFWs - that is just standard firewall from the days of the first packet filters..

Not in Cisco. By default, a Cisco 1920 router, which has a 1st gen firewall built in to it in the form of ACLs, will pass all packets, including ICMPs by default. You have to configure the device to with ACEs to tell it what traffic to permit or not permit. The standard ICMP block wasn't until late in the game of 2nd gen firewalls when the ASA became the de-facto.

[grunger106]

4 minutes ago, aus73 said:

Not true. The need for routing is determined by subnets. While the router or firewall does know about the networks connected directly to it, if the logical networks are different (i.e. - the subnets are different) the device does not forward the traffic between them UNLESS you tell the device to do that. In traditional L3 devices, that means a route has to be established either via a static route or one of the routing protocols (RIP/RIPv2/RIPng, OSPF, EIGRP). In an NGFW, the security policy does the routing. And in 1st and 2nd gen firewalls, they are really just packet filters; so, they don't do any forwarding whatsoever. The 1st gens just looked at source, destination, and port, and it did what the first ACE that matched the traffic told it to do, either permit or deny, in a top down search. If it didn't find a match, the packet was dropped by default. 2nd gens did the same thing as well as look at the state of the packet (established or not), and was able to make decisions on whether to drop or permit based on that. In both cases, if the packet was permitted, it still needed a separate device to route.

I think that was a response to a example I'd given where I'd failed to quantify the exact environment I was talking about and may have slightly misread the OP looking back at it now.

 

I have an Sophos XG as which does filtering, firewall, NAT etc and is the final hop on my LAN before the outside world
This is connected on its own /30 network that exists purely to connect it to my L3 switch.

The L3 switch has a single static route to 0.0.0.0 with the XG as the next hop

The XG has static routes to each established network (that need internet access) on the switch, with the switch as the next hop

My VLANs are separated via ACLs at the switch, the XG has no knowledge of them hence the need for the routes

 

However the L3 switch doesn't need static routes to its connected networks obviously, which I think where I might not have made myself that clear

 

On that note - while the HP 1920 is a decent switch for the money, the ACL implementation method is beyond awful (really awful) and it only support static routes as its L3 Lite.

[+BudMan MVC]

55 minutes ago, aus73 said:

Cisco 1920 router,

Lets be clear about the "router" in the name there... Its a ROUTER out of the box and doesn't do any sort of firewall unless you enable those features and turn it on... That is completely different than a "firewall" that also routes.

 

Pfsense which was part of the discussion, while sure it will route between segments you create. And the default lan has any any rules on it.. Any new interface you create has NO rules and will not pass traffic - until y you create the rules to allow it.

[+BudMan MVC]

1 hour ago, aus73 said:

that means a route has to be established either via a static route or one of the routing protocols (RIP/RIPv2/RIPng, OSPF, EIGRP)

This is also not true.. Out of the box anything doing L3 routing will inherently know how to route between two connected networks..

 

If on a "routing" device I have 192.168.1/24 and 192.168.2/24 connected to it - and you send something to its IP on 1, it knows how to send that to 2/24 - be it allowed or not would depend on any firewall rules you enabled on this "routing" device.