[NCLUG] how was I hacked?

[Daniel Herrington]

  I got a couple of strange email messages during the past week from my 
web server:

First message:

 ################## LogWatch 2.1.1 Begin ##################### 


 --------------------- ModProbe Begin ------------------------ 

Can't locate these modules:
   0: 1 Time(s)


 ---------------------- ModProbe End ------------------------- 



 ###################### LogWatch End ######################### 



Second message:

/etc/cron.weekly/makewhatis.cron:


zcat: ./.../psyBNC2.2.2.tar.gz: unexpected end of file



Needless to say, I was somewhat disturbed by these, since normally I 
would get an empty email from LogWatch each morning. I then did a 
"locate psyBNC", which returned the following:

/usr/man/man8/.../psyBNC2.2.2.tar.gz

Interesting. I went to this directory, and now I see this:

drwxr-xr-x   2 root     root         4096 Apr  6 07:25 ./
drwxr-xr-x   3 root     root         4096 Apr  6 07:24 ../
lrwxrwxrwx   1 root     root           11 Apr  6 07:24 .1addr -> /lib/defs/q
lrwxrwxrwx   1 root     root           11 Apr  6 07:24 .1boot -> /lib/defs/l
lrwxrwxrwx   1 root     root           11 Apr  6 07:24 .1file -> /lib/defs/r
lrwxrwxrwx   1 root     root           11 Apr  6 07:24 .1logz -> /lib/defs/s
lrwxrwxrwx   1 root     root           11 Apr  6 07:24 .1proc -> /lib/defs/p
-rw-r--r--   1 root     root       356532 Apr  6 07:25 backup.tar.gz
-rwxr-xr-x   1 root     root        16070 Apr  6 07:24 dogsnif*
-rwxr-xr-x   1 root     root          613 Apr  6 07:24 get-eggdrop*
-rwxr-xr-x   1 root     root          196 Apr  6 07:24 get-glibc*
-rwxr-xr-x   1 root     root           54 Apr  6 07:24 get-psy*
-rwxr-xr-x   1 root     root          161 Apr  6 07:24 get-wget*
-rwxr-xr-x   1 root     root         7287 Apr  6 07:24 mirkclean*
-rwxr-xr-x   1 root     root          244 Apr  6 07:24 patch_ssh*
-rw-r--r--   1 root     root         4019 Apr  6 07:24 psyBNC2.2.2.tar.gz
-rwxr-xr-x   1 root     root         8840 Apr  6 07:24 redo*
-rwxr-xr-x   1 root     root         9271 Apr  6 07:24 rh_patch*
-rwxr-xr-x   1 root     root         1345 Apr  6 07:24 sauber*
-rwxr-xr-x   1 root     root        30656 Apr  6 07:24 synscan*
-rwxr-xr-x   1 root     root        10368 Apr  6 07:24 z0ne*
-rwxr-xr-x   1 root     root        15800 Apr  6 07:24 zxsniff*

So obviously, the server has been hacked! This is very irritating. Now I 
need to fix it, but I have questions about what happened:

Is anyone familiar with this type of hack? If so, was it probably done 
through some service I should've had turned off, or some other way?

This server is running RedHat 7.2. I admit I've been lazy about 
installing security fixes. Am I better off just installing RedHat 8.0 or 
9 and their corresponding security fixes, or is there an easy way to fix 
the current installation?

Also, this server sits behind a firewall router that only allows web and 
ssh ports through. I thought this was pretty safe, but apparently not 
safe enough?

Thanks for any help,
Daniel Herrington