[NCLUG] using Bayesian networks for simple systems debugging?

[Bob Proulx]

Matt wrote:
> At the same time there must be hundreds of thousands of networks that
> are very similar to mine and all of these people have to learn this
> stuff the hard way.

The problem is that the easy cases are already easy.  Take setting up
a dns server for example.  At the danger of starting up a mine is
better than yours but wanting to provide a concrete example, on a
Debian system:

  sudo apt-get install bind9

There you go.  A fully installed and configured caching dns server is
now installed and configured.  The easy case is very easy.  Want to
install NTP?  This installs and configured ntp.

  sudo apt-get install ntp

Again very easy.  The common case that is the same as a thousand other
people is already a solved problem.  Good initial guesses can be made
by the packager to create a good enough default out of the box for
most cases.  The problem you must be running into are the not so
common cases.  And even for the above examples they won't work for all
environments by default and will for some need special configuration
such as for those behind restrictive firewalls.

Lets say that you want to install and configure your own DHCP server.
This is very common and yet at the same time 99% of the computer using
population does not even know what that is, does, or means.  Much of
the public would think that DHCP servers are something that only an
ISP would ever want to do if they think about it at all.  And yet
almost every computer using home today has one in a dedicated firewall
router and they don't even know it.

But in any case, after the dhcp server software is installed now there
is the problem of configuring it.  This could mean a zillion things.
Are you using a 10.* local network IP address or a 192.168.* network
address?  Or even possibly others.  Which subnet in that range?  Are
you wanting to assign fixed IP addresses?  Are you wanting to do PXE
or tftp boot of diskless clients?  Are you wanting to assign a
specific domain name for dns lookups?  Are you wanting to have clients
register themselves in a dynamic dns zone?  Are you wanting to use
multiple redundant dhcp servers for high reliability in the case that
one of them is offline?

Suddenly the case becomes quite complicated.  What is the right answer
to those questions?  One problem is that there is no single right
answer to those questions!  There are valid reasons for different
people to answer each of those differently.  This makes for a very
large combination of possible right answers.  Which one is the right
answer for you?  If you don't know then how can anyone else know?

If I were to install and administer your system I would give you *a*
configuration that I thought was best.  It would be based upon the
best knowledge that I would have beforehand.  And I would be wrong
most of the time.  If I assumed 192.168.* I would be wrong for the
10.* folks and the reverse.  If I assumed 192.168.1.255 I would be
wrong for the people with two of those and had been pushed up to
192.168.2.255 for the second one.  If I assumed always a dynamic pool
then I would be wrong for people needing to set up a server with a
fixed IP address.  And so on.  I think it is impossible to
preconfigure this information to be right even more than 50% of the
time.  It is a very hard problem.

> Instead, how about using some form of knowledge extraction technique
> that would examine all the configuration files across a lot of
> systems to generate something that could look at a specific
> configuration and let you know how it differs from the closest
> configuration that works?

I don't think this would work because there will be differences that
are not due to errors in your configuration but due to differences in
the surrounding environment.  I don't know how you could separate one
from the other.

> If Bayesian networks are used to determine if a message is spam
> couldn't it be used to determine if, say, nfs on a small system is
> configured correctly? I'm guessing the idea in general would work,
> but I'm wondering about the details.

I would be worried about the details too.  Is this NFS on a private
network where security is not a concern?  (Of course NFS always means
a lower security level so that is a redundant statement.)  In which
case that almost certainly also means configuring NIS/YP too because
with NFS uid numbers must match across systems.  What would the YP
domain name be for the configuration of NIS/YP?  Often the same as the
dns domain name for simplicity but also often something unrelated.

If a Bayes engine saw that NFS was enabled but NIS/YP was not enabled
statistically that would almost certainly indicate an error.  But
would it always?  Not necessarily.  Perhaps password database
information is shared using LDAP or possibly by actually sync'ing
/etc/passwd files.  If Bayes sees that one system uses async nfs while
another uses sync which one is in error?  If Bayes sees that one
system exports / and /home while another only exports /mnt/data which
one is in error?

I think it might be possible if the number of learning cases for the
Bayes engine were very large.  Possibly large enough to encompass all
of the existing configurations.  But the database to hold the
information would need to be equally large.  And there are privacy and
security concerns since it would be difficult to collect that
information.  And the value of Bayes is to extrapolate from a small
training set into a larger repeated case catagorization.  If all cases
would need to be learned then the value is much reduced.

> I doubt if something like this would work for any organization that
> hires systems administrators, but could it work for people with small
> systems? Think of the people with no computer knowledge but still have
> a network at home.

I hate to be sounding all poo-poo over this idea and so regardless of
my being skeptical I encourage you to keep going.  I think it would be
really cool if you gave your idea a try and then reported back how it
is working out.  I love it when people push through and succeed doing
something that other people told them would not work and it would not
bother me at all to be on the receiving end of that.  But you were
asking for an opinion and I won't be shy about providing feedback.
And perhaps along the way you would learn a different technique or
learn a different use for this technique.  Often the best inventions
were on the way to inventing something different.

Bob