tyler.durden
/
pandorafms
mirror of https://github.com/pandorafms/pandorafms.git
1
0
Fork 0
Code Issues Packages Projects Releases Wiki Activity
pandorafms/pandora_doc/en/pandora_introduction.xml

510 lines
21 KiB
XML
Raw Blame History

<?xml version="1.0" encoding="utf-8"?>
<chapter>
<title>Introduction to Pandora FMS</title>
<sect1><title>Pandora. The Free Monitoring System</title>
<para>
Pandora FMS is a monitoring application to watch systems and
applications. Pandora allows to know the status of any element of
your bussiness systems. Pandora watch for your hardware, your
software, your multilayer system and of course your Operating
System. Pandora could detect a network interface down and the
movement of any value of the NASDAQ new technology market. If you
want, Pandora could sent a SMS message when your systems
fails... or when Google value low below US$ 330.
</para>
<para>
Pandora FMS will adjust, like an octopus, to your systems and
requirements, because it has been designed to be open, modular,
multiplattform and easy to customize.
</para>
</sect1>
<sect1>
<title>Introducing Pandora FMS.</title>
<para>
&pandora; is a monitoring tool that allows a system
administrator to visually analise the status and efficiency of
Operating Systems, Servers, Applications and Hardware Systems -
such as firewalls, proxies, databases, Web servers, tunnelling
servers, routers, switches, processes, services, remote access
servers, etc. - all integrated into an open and distributed
architecture. Pandora can be implemented on top of any operating
system, with specific agents for each platform. Pandora can also
monitor any TCP/IP hardware system, as load balancers,
routers, switches, printers, etc.
<graphic fileref="images/esquema.png" scale="50" align="center"/>
Pandora architecture is formed of four main components:
<itemizedlist mark='bullet'>
<listitem>
<para>
<emphasis>Web Console</emphasis>: Pandora's user
interface. The user controls and operates the system with
it. Several Web consoles can be implemented in a single
system. The Web console is written in PHP, and it lies on top
of a database and a Web server. It is compatible with any
platform - GNU/Linux, Solaris, Win2000, AIX, etc.T
official supported platform is GNU/Linux, though
</para>
<para>
The console permits the user to control the status of the
agents, view statistical information, generate graphs and
data tables, keep a system incident control,moreover it is able
to generate reports and change the alerts, agents, and user
profile settings.
</para>
</listitem>
<listitem>
<para>
<emphasis>Server</emphasis>: In Pandora 1.2 there are three
different servers:
</para>
<para>
The core server is the receptor of the data packages and
generates the alerts - it is the brain of the system.
Several servers can work alongside for larger systems.
The core server accesses Pandora database, which is shared
with the Web server, and stores the processed data
packages. Server executes as daemon, and processes the
packages stored in its file system. Data is generated by the
system agents. Despite the server's low system resources
comsumption and simple installation and operation, the core
server is the most critical element of the system. The core
server receives and processes the produced data, and fires
the alerts and the events.
</para>
<para>
The Network Servers monitorize remote systems using
network resources like ICMP, TCP, UDP or SNMP
Queries. Network Servers are acting themselves as "Network
Agents". This server fires the alerts and the events for this
modules.
</para>
<para>
The SNMP Server receives and processes the snmp traps, and fires
the alerts associated to it.
</para>
</listitem>
<listitem>
<para>
<emphasis>Central Database</emphasis>: At the moment the
system only supports MySQL. The central database keeps all
the information Pandora needs to work - agent data,
settings, user information, incidents, system settings,
etc. The system can use a MySQL cluster to store the
information, or a High Availability (HA) solution for larger
sytems.
</para>
<para>
This database can work with any of the platforms officially
supported by MySQL. Pandora can be implemented with MySQL
versions from 3.0 to 5.0, although the latest one is recommended.
</para>
</listitem>
<listitem>
<para>
<emphasis>Pandora Agents</emphasis>: They collect all the
system's data. They are executed in each local system,
although they can also collect remote information by
installing monitoring sytems for the agent in several
different machines - called satellite agents.
</para>
<para>
They have been developed to work under a specific platform,
making use of the specific tools of the used language:
ShellScripting for Unix - which includes GNU/Linux, Solaris,
AIX, HP-UX and BSD, as well as the Nokia's IPSO. Pandora
agents can be developed in virtually any language, given its
simple API and being open source. Windows agents are
developed in a free development environment for C++ and uses
the same interface and modularity than Unix agents.
</para>
<para>
The old agent for Windows plattforms was developed on VBS
Scripting language, and is deprecated with the new Pandora
1.2 windows agent.
</para>
</listitem>
</itemizedlist>
<graphic fileref="images/pandora_arch1.jpg" scale="70" align="center"/>
</para>
</sect1>
<sect1><title>What kind of systems/ services can be monitored?</title>
<para>
At present, with Pandora any process or system that through a
command returns a value can be monitored, as well as any value in
any Operating System log file or similar. Some examples of already
existing implementations below:
<programlisting>
Number of connections (sessions) of Checkpoint FW-1
Number of NAT sessions of Checkpoint FW-1
Number of connections of Linux NetFilter / IPTables firewall
Number of FW-1 logged packets
Number of FW-1 dropped packets
Number of FW-1 accepted packets
State of High Availability in FW1 NG
Last policy installed in a Firewall-1 module
Synchronization state of the modules in FW1 NG
CPU of the system: idle, user and system
Number of processes of the system
Temperature of the CPU of a system
Value of a MS Windows registry entry
Queued jobs in a generic dispatcher
Memory of the system: free, swap, kernel Fw-1, cache
Percentage of free space on disc (for different partitions)
Messages processed by a mail gateway
Existence of a string in a text file
IP traffic (filtering based on the connections of the firewall)
Hits of pages in HTTP Servers (Apache, iPlanet, IIS, Netscape)
Percentage of erroneous packets in a Gateway
Connections established in a Remote Access Server (RAS)
Size of a file
Open sessions by a VPN server
MySQL Performance: Threads, queries, sessions...
Snort system state
Reported events by IDS (Snort) up to six levels of priority
Network load
Number of local Connections (TCP, UDP, Unix sockets)
Detected viruses by a Web Antivirus Gateway
ICMP latency time towards a host
Rate of average transference in a file transfer tool
Number of DNS requests attended by a server (including types)
Number of FTP sessions attended by a FTP server
(Generic) State of any active process / service in the system
(Generic) State of any countable parameter of the system
</programlisting>
</para>
<sect2><title>Global architecture</title>
<para>
Pandora 1.2 has changed many things from 1.1 version, but this
graph representing Pandora architecture is very useful to
understand in a single graph, all components.
<graphic fileref="images/pandora_arch2.jpg" scale="45" align="center"/>
</para>
</sect2>
</sect1>
<sect1><title>Information gathering with Pandora agents</title>
<para>
Pandora agents are based on native languages in every platform:
scripts that can be written in any language. It's possible to
reproduce any agent in any programming language and the existing ones
can be extended without difficulty in order to cover aspects
not taken into account up to the moment.
</para>
<para>
These scripts are formed by modules each one gathering a
"chunk" of information. Thus, every agent gathers several "chunks"
of information; this one is organized in a data set and stored in
a single file, called data file.
</para>
<para>
The process of transferring the data file from the agent to the
server is made regularly at a defined time interval in the agent
configuration file, pandora_agent.conf. It's possible to modify
that parameter in order to do not fill the database with non-relevant
information, either load the network or affect the system
performance. The default interval is 300 (seconds), which is
equivalent to five minutes. Values under 100 (seconds) are not
recommended since host performance could be affected, besides
loading excessively Database and the Operating System of Pandora
Server. Pandora is not a real time system; it's an applications
and systems general monitoring system in environments that are not
critical at real time.
</para>
<para>
Packets transfers are made via SSH, with DSA authentication
(although also RSA can be used). The process is completely safe
since neither any password nor unencrypted confidential
information is sent. Confidentiality, integrity and authentication
of the connections between the agent and the server are
ensured. In the Agents and Server Installation and Configuration
guides, the process of generation of keys to do the automatic SCP
transfer is detailed.
</para>
<para>
Also the transfer via FTP or any other file transfer system could
be made, although SSH has been chosen for security and
compatibility with most of the systems in the market.
</para>
<para>
Pandora Agents are thought to be executed from the agent from
which they gather information, although the agents can gather
information of accessible machines from the host where they are
installed. In this case those agents are called "Satellite
Agents". These Satellite Agents can use Telnet, SNMP or any other
commands to get the information.
</para>
<para>
We can also have a host with several agents: Some that gather
information from the accessible machines (acting as "satellite
agents") and the Standard Agent that monitors the host where it's
running.
</para>
<sect2><title>XML Data files</title>
<para>
The data file has the following syntax:
<programlisting>
hostname.serialnumber.data
</programlisting>
This is an XML file, and its name is the combination of the
hostname where the agent runs, a different serial number for every
data package and the extension .data that indicates that it's a
data file.
</para>
<para>
We also have a control file for every data file:
<programlisting>
hostname.serialnumber.checksum
</programlisting>
This file has .checksum extension and contains a MD5 hash of the
data file. This allows checking that the information has not been
changed before being processed.
</para>
<para>
The XML data file generated by every agent is the core of
Pandora. This file has the information gathered by the Agent. Its
easy structure allows that any user could create its own
developments to be processed in Pandora, or use the included ones.
An example of the information included into the data file below:
<screen>
<![CDATA[
<agent data os_name="SunOS" os_version="5.8" timestamp="300"
agent_name="pdges01" version="1.0">
<module>
<name>SSH Daemon</name>
<type>generic_proc</type>
<data>1</data>
</module>
<module>
<name>FTP Daemon</name>
<type>generic_proc</type>
<data>0</data>
</module>
<module>
<name>DiskFree</name>
<type>generic_data</type>
<data>5200000</data>
</module>
<module>
<name>UsersConnected</name>
<type>generic_data_inc</type>
<data>119</data>
<min>1</min>
<max>250</max>
<description>Users currently connected</description>
</module>
<module>
<name>LastLogin</name>
<type>generic_data_string</type>
<data>slerena</data>
</module>
</agent_data>
]]>
</screen>
</para>
</sect2>
<sect2><title>Pandora servers</title>
<para>
With Pandora 1.2 version, you have three different types of servers:
<itemizedlist mark='bullet'>
<listitem>
<para>
<emphasis>Pandora Data Server</emphasis>. This is a PERL
application that that processes the information sent by the
agents. The agents send the XML data file via SSH and the
server periodically verifies if it has new data files
waiting to be processed. You can setup different data
servers in different systems or in the same host (that will
be different virtual servers).
</para>
</listitem>
<listitem>
<para>
<emphasis>Pandora Network Server</emphasis>. This is a PERL
application that execute network tasks like sending pings,
TCP requests, SNMP requests and UDP request. When you assign
an agent to a server, you are assigning to a network server,
not a data server, so, this is very important that machines
running network servers have "network visibility" to hosts
assigned in network modules.
</para>
<para>
For example, if you create a module to make a ping check to
192.168.1.1 and assign this agent/module to a server in a
192.168.2.0/24 network without access to 192.168.1.0/24
module will always report DOWN.
</para>
</listitem>
<listitem>
<para>
<emphasis>Pandora SNMP Server</emphasis>. This is a PERL
application that parse output from standard snmptradp (we
provide one binary for snmptrapd, but it is possible that
you need to replace it with a binary that runs better in your
system). This daemon receives SNMP traps, and Pandora SNMP
Server stores in database and fire alerts assigned in
Pandora SNMP Console.
</para>
</listitem>
</itemizedlist>
</para>
<para>
Data are extracted from the data file, identifying origin, type
and category. Once it's classified, the data are inserted into the
Database by the same Perl script.
</para>
<para>
Pandora Server can work in High Availability and/or Load
Balancing. In a very big architecture, several Pandora Servers can
be arranged simultaneously in order to be able to manage big volumes of
information distributed by geographical or functional zones.
</para>
<para>
Pandora Server is always running (as a daemon) and permanently
verifies if some element causes to fire an alarm. If so, it
executes the action defined in the alarm, as to send a SMS, an
email, even activates the execution of a SCRIPT or to send an HTTP
form.
</para>
<para>
We could have several simultaneous servers, one of them is the
Main Server or "Master Server " and the rest of servers are "Slave
Servers". The Master Server is the only one that verifies the
alarms if any agent goes down. The server which receives the data
file from the agent always fires the rest of alarms, defined in
the agents' modules. This is also important if this server changes
(due to configurations of high availability, load balancing or
clustering).
</para>
</sect2>
<sect2><title>Pandora console</title>
<para>
The Web Console is a web application that allows to see
graphical reports, state of every agent, also to access to the
information sent by the agent, to see every monitored parameter
and to see its evolution throughout the time, to form the
different nodes, groups and users of the system. It is the part
that interacts with the final user, and that will allows you to
administer the system.
</para>
<para>
The Web Console is written in PHP and no plug-in, Flash, Java or
ActiveX is needed to access the console, only a browser that
supports HTML and CSS (IE5+ or Mozilla 4+). Pandora Web Console can
run in several servers, the only thing you need is to be allow to
access Pandora Database, where Pandora stores all the information.
</para>
</sect2>
<sect2><title>Pandora database</title>
<para>
Pandora uses a SQL Database to store all the information. Pandora
maintains an asynchronous database with all the received data,
making a temporary cohesion of everything it is receives and
normalizing all the information from the different sources. Every
Agent data module generates an entry of information for every data
bundle, which implies that a real production system can have of
the order of ten million of data, or information atoms.
</para>
<para>
This information is managed automatically from Pandora, carrying
out a periodic and automatic maintenance of the database. This means
that there is no operator either manager required to run tasks as database
administration ones. This is possible thanks to a periodic purge of the past
information over a date (by default 90 days), as well as a data
which is older, by default, 30 days.
</para>
<sect3><title>Compacting data</title>
<para>
Data stored by Pandora are useful to see evolutions through
the time, in order to: make statistics, generate reports and to do
capacity planning, as well as other statisticals tasks.
To do that it isn't necessary to have all the data, but it's
enough to have a representative sample, of smaller resolution,
enough to carry out the task that is needed.
</para>
<para>
With that philosophy the compaction system has been
constructed. For instance, If we have a sample of 9.000 elements,
distributed during 90 days, Pandora will take the data of
last month, which would be 3.000 elements and will compress it in 300.
In the graphs they will practically be equal, and it will be useful for the reports, statistics
and other tasks. This is made thanks to a interpolation in temporary
strips, in a totally automatic and periodic way, there is no user
or the administrator needed to do this.
</para>
</sect3>
</sect2>
</sect1>
<sect1>
<title>Pandora 1.2 new features</title>
<para>
<emphasis>Alert system</emphasis>. Now it is possible to define a
"minimun" and "maximum" limit to fire an alert, just to delete
"noisy" data that fires false positives.
</para>
<para>
<emphasis>Network Subsystem</emphasis>. Now it is possible to
monitor and analyze data using remote network tools, without using
agents, from the new Pandora Network Server component. All
management are made from Pandora Console, and now you will be able
to make ICMP checks (Ping), size network latency, get all types of
SNMP values (including scanning MIB), and makes TCP/UDP
connections to check ports, and test text applications, sending
texts and waiting for a specific response.
</para>
<para>
<emphasis>Module groups.</emphasis>Modules now could be grouped
using a new "module groups".
</para>
<para>
<emphasis>Network data refresh on demand.</emphasis> Could
be for each module or using a "global group refresh", forcing
Pandora Network Servers to refresh all network modules inside a
group.
</para>
<para>
<emphasis>Online contextual help</emphasis>, for Pandora WEB Console.
</para>
<para>
<emphasis>New Pandora server infraestructure.</emphasis>
</para>
<para>
<emphasis>New SNMP trap console</emphasis> to receive SNMP traps
and assigning alerts.
</para>
<para>
<emphasis>Internal messaging system</emphasis>, to notify events
to Pandora users.
</para>
<para>
<emphasis>Agent detail view autorefresh</emphasis>
</para>
<para>
<emphasis>New main agent group view</emphasis>
</para>
<para>
<emphasis>Improved database management system</emphasis>, that
allows to manage much more data.
</para>
</sect1>
<sect1>
<title>About Pandora
</title>
<para>
Pandora is a project initiated and mainly developed by Sancho
Lerena, at present other people is working on it: Raul Mateos,
David Villanueva, Esteban Sanchez, Jose Navarro, Jonathan
Barajas and Manuel Arostegui. We want to thank many other people who help us
with translation, graphic design, bugs reporting and interesting
ideas.
</para>
<para>
Pandora is Free Software, and is published under GPL Licence. In
order to know the last features, go to the official web site of
the project in http://pandora.sourceforge.net.
</para>
</sect1>
</chapter>
Reference in New Issue View Git Blame Copy Permalink