Author Topic: SSG5 suddenly stopped outgoing sessions  (Read 18371 times)

muppet

  • Global Moderator
  • Full Member
  • *****
  • Posts: 211
  • Karma: +0/-0
  • I Like Beer
    • View Profile
    • LiCe for EPIC5
Re: SSG5 suddenly stopped outgoing sessions
« Reply #20 on: February 05, 2008, 02:01:59 am »
If the sessions aren't timing out, they must be in use.

ssh to the console of the device and do a "get session" - You can then examine in detail what the ports are.

Most probably someone running Skype or Bittorrent.


echo

  • Full Member
  • ***
  • Posts: 135
  • Karma: +2/-0
  • I'm in the background
    • View Profile
    • 3<-0!493
Re: SSG5 suddenly stopped outgoing sessions
« Reply #21 on: February 05, 2008, 02:27:06 am »
Thanks but I've done that already with Tim's Analyzer. I scanned the computer which had most connections - no spyware or viruses - or thay weren't detected. The user who uses that machine is a quiet elder woman. I didn't detect any torrents installed in that machine. And the best thing is that she has Skype installed, but never uses it! And still 272 sessions, yea... And shuts computer down after work every day. Really odd. There is a scheduled task by which Spybot Search & Destroy updates and scans machine every friday, but I couldn't detect that that was the creator of UDP-s.
_ ____ _____ _ ____ __ ___
___ ___ __ ________ ___ __
            echo.planet.ee

muppet

  • Global Moderator
  • Full Member
  • *****
  • Posts: 211
  • Karma: +0/-0
  • I Like Beer
    • View Profile
    • LiCe for EPIC5
Re: SSG5 suddenly stopped outgoing sessions
« Reply #22 on: February 05, 2008, 02:31:24 am »
Sorry you're right - I didn't read the full thread and I will bash myself on the head accordingly.

The next step to troubleshoot this if I was in your shoes would be to install Wireshark on the machine in question and leave it logging in the background for a couple of hours.  Clear the sessions on the firewall (clear session) and see if they reappear.

If they do, have a look at your wireshark capture and see if you can figure out what the traffic is!  If it doesn't appear in Wireshark, then make sure you

a) Don't have a rootkit installed
b) You really have the right machine (check the arp addresses "get arp" to make sure there's not multiple machines with the same IP)

Hope this helps more than my last boneheaded post :)

echo

  • Full Member
  • ***
  • Posts: 135
  • Karma: +2/-0
  • I'm in the background
    • View Profile
    • 3<-0!493
Re: SSG5 suddenly stopped outgoing sessions
« Reply #23 on: February 23, 2008, 01:28:27 pm »
Thanks muppet for letting me know about wireshark! My problem suddenly stopped so I didn't try it on that comupter, but on mine :)

Anyway, after some 10 days of (Juniper) working nothing bad happened. Then I made restart, and at the moment it has worked for 9 days and there were never more than 200 sessions in that time, usually around 100. I can't understand. The only thing I did on that one computer I mentioned above was that I manually ran the Spybot Search & Destroy which was scheduled to run in Fridays, and it found nothing to remove except one thing: with group policy or what there were some notifications about security problems just removed - nothing special. And I remind that it has been before that outgoing traffic slowed down more than once a week, not just after Fridays, when Spybot perhaps couldn't automatically download new definitions and hanged somehow.

At the moment in Saturdays night it has ran for 9 days 9 hours and it has only 16 sessions.

But... Yesterday I replaced a Zyxel Zywall 5 with Juniper SSG5 in another company and things were worse! Within hours the session table was so full that outgoing traffic just didn't exist anymore. When I pinged some public IP from inner network I didn't get anything, but when I pinged from SSG5, it pinged with 100% success.

I made the session analyze with Tim's good tool and it also turned out that more than 99% was UDP and there were 6 machines where from the traffic came from. Disabling Trust->Untrust default traffic for some seconds made everything clear, and then the number of sessions started going up pretty fast again (about 10 sessions per 1 second) and then slowed down a bit. After hours (half an hour ago) I made "session clear src-ip <ip>" for all the "best" six machines (when SSG5 had 1700 sessions and the best of them 461!) and it looks like it is going to start again, but not that fast.

I searched about Skype and Juniper but didn't find nothing special. Only that there was exactly the same case somewhere in a hotel when a client opened his computer with Skype windows open and immediately his MAC was blocked by hotels firewall because that Skype UDP traffic looked like an attack from inside.

Could it be that Skype as an example of P2P program, takes down Juniper SSG5? Because when Skype keeps running, whoever else can use this company's internet resource to redirect Skype traffic (by pinching holes into firewall as it does)?
« Last Edit: February 23, 2008, 01:30:41 pm by echo »
_ ____ _____ _ ____ __ ___
___ ___ __ ________ ___ __
            echo.planet.ee

greg1c

  • Full Member
  • ***
  • Posts: 190
  • Karma: +0/-0
    • View Profile
Re: SSG5 suddenly stopped outgoing sessions
« Reply #24 on: February 23, 2008, 01:33:56 pm »
You could limit the the number of sessions from one ip using the screening options (best option), you could limit bandwidth (although this will do nothing to sessions) for that protocol.  You could change the UDP flood options lower, to automatically block this UDP traffic once it gets to so many PPS.  You could get the extended license and double the sessions of your SSG 5 to 8,192.

Greg

echo

  • Full Member
  • ***
  • Posts: 135
  • Karma: +2/-0
  • I'm in the background
    • View Profile
    • 3<-0!493
Re: SSG5 suddenly stopped outgoing sessions
« Reply #25 on: February 23, 2008, 02:27:46 pm »
Hi greg1c, big thanks for the tips! In trust zone I set the source IP-based session limit to 50 sessions and we'll see, what happens. Also in trust zone I lowered the UDP flood setting to 500 PPS, but I'm not quite sure how much this usually is when UDP traffic is normal. About number of sessions - it already has 8064 sessions available with 6.0.0r3.0 (with 5.4.0 it has 4064 as much as I know). Did you mean that it may have some 16000 sessions with appropriate license?
_ ____ _____ _ ____ __ ___
___ ___ __ ________ ___ __
            echo.planet.ee

echo

  • Full Member
  • ***
  • Posts: 135
  • Karma: +2/-0
  • I'm in the background
    • View Profile
    • 3<-0!493
Re: SSG5 suddenly stopped outgoing sessions
« Reply #26 on: February 26, 2008, 02:30:23 am »
Well, that screening option to limit source-ip based sessions helped to let the firewall running yesterday so that the session table didn't grow too large. I raised the limit from 50 to 100 because is saw that even tcp port 80 was limited many times.

But yea, the alarm log is completely full (2047 logs) with announcements that the src-ip session limit was reached. But no complaints so far about bad internet connection, bad skype quality or smth...
_ ____ _____ _ ____ __ ___
___ ___ __ ________ ___ __
            echo.planet.ee

ahfaris

  • Newbie
  • *
  • Posts: 3
  • Karma: +0/-0
    • View Profile
Re: SSG5 suddenly stopped outgoing sessions
« Reply #27 on: February 26, 2008, 08:51:16 am »
See the routing for for 0.0.0.0/0 at networking - > routing -> destination ,, it must exist and point to your router or default gateway .

echo

  • Full Member
  • ***
  • Posts: 135
  • Karma: +2/-0
  • I'm in the background
    • View Profile
    • 3<-0!493
Re: SSG5 suddenly stopped outgoing sessions
« Reply #28 on: February 26, 2008, 08:56:41 am »
Yea, it certainly does exist! If it doesn't, then there would be no internet availability for that company at all. Actually, internet from inside out, because from outside in it may work well even without that routing.
_ ____ _____ _ ____ __ ___
___ ___ __ ________ ___ __
            echo.planet.ee

JuniperGuy

  • Newbie
  • *
  • Posts: 20
  • Karma: +0/-0
    • View Profile
Re: SSG5 suddenly stopped outgoing sessions
« Reply #29 on: February 26, 2008, 07:59:01 pm »
I think ahfaris is referring to a "set route 0.0.0.0/0 gateway x.x.x.x"  statement, or lack thereof. I did see the "set interface ethernet0/0 gateway E.F.G.209" statement, (I had a hard time digging it up) it is a legacy command from ScreenOS 3.0 and was dropped when 4.0 come along.

Issues with command are
# Packets don't get routed out untrust interface properly
# Packets routed out default route instead of specific route

Here is a KB article explaining that you should use a route statement:
http://kb.juniper.net/CUSTOMERSERVICE/index?page=kbdetail&record_id=S:0244022611e8310108012c3c1906d9d

echo

  • Full Member
  • ***
  • Posts: 135
  • Karma: +2/-0
  • I'm in the background
    • View Profile
    • 3<-0!493
Re: SSG5 suddenly stopped outgoing sessions
« Reply #30 on: March 03, 2008, 03:17:44 am »
Hmm, I had noticed that there is a difference about default route when one does it during initial configuration or after logging in and then doing it in Network->Routing->Destination. The "old" one appears when you do it with initial configuration, the "new" one when you set the route via webui after logging in. That's why it is there. That article referred to Netscreens, not Juniper SSG5 and screenos 4.0, not 6.0... Don't know, what to think about that now.

Anyway, even after taking down the src-ip session limit and replacing test-manageable switch with their old one the problem hasn't appeared anymore.

But there's still a problem with SSG-140 and that's probably not the session issue - some 400 sessions out of 48 000 possible can't make Juniper slow, I think. If it appears again and I can't find solution, I have to start a thread about it.
_ ____ _____ _ ____ __ ___
___ ___ __ ________ ___ __
            echo.planet.ee

rowlandg

  • Newbie
  • *
  • Posts: 35
  • Karma: +0/-0
    • View Profile
Re: SSG5 suddenly stopped outgoing sessions
« Reply #31 on: March 07, 2008, 12:49:54 pm »
we had an issue on 6.0.2 and 6.0.3 where the box would either stop matching policies or stop NATTING and put them in the global cleanup rule even though for the last hour the same devices where working. Only a reboot fixes this or an upgrade to 6.0.4

Another customer had a buffer issue when multiple 10k 20k or 50k files going through ok but if they went to 60k - 100k it would work for 30mins then just give up. We checked/adjusted the MSS and MTU settings and this made no difference we upgraded wednesday and the customer has not had any issues andthey have put through loads.
!!!6.1 you can limit sessions per policy  so you could use this with some QOS