Merge pull request #1968 from ktsaou/master

[netdata.git] / README.md

diff --git a/README.md b/README.md
index c44b3efd5319b031dff4f3f0bffe0b26ce3a8126..3d73b9ecce452c2a9c340e71990c5014501f133a 100644 (file)
--- a/README.md
+++ b/README.md
@@ -1,17 +1,25 @@
-# netdata [![Build Status](https://travis-ci.org/firehol/netdata.svg?branch=master)](https://travis-ci.org/firehol/netdata) [![Coverity Scan Build Status](https://scan.coverity.com/projects/9140/badge.svg)](https://scan.coverity.com/projects/firehol-netdata) [![Docker Pulls](https://img.shields.io/docker/pulls/titpetric/netdata.svg)](https://hub.docker.com/r/titpetric/netdata/)
-
+# netdata [![Build Status](https://travis-ci.org/firehol/netdata.svg?branch=master)](https://travis-ci.org/firehol/netdata) [![Coverity Scan Build Status](https://scan.coverity.com/projects/9140/badge.svg)](https://scan.coverity.com/projects/firehol-netdata) [![Codacy Badge](https://api.codacy.com/project/badge/Grade/a994873f30d045b9b4b83606c3eb3498)](https://www.codacy.com/app/netdata/netdata?utm_source=github.com&utm_medium=referral&utm_content=firehol/netdata&utm_campaign=Badge_Grade) [![Code Climate](https://codeclimate.com/github/firehol/netdata/badges/gpa.svg)](https://codeclimate.com/github/firehol/netdata)

 > *New to netdata? Here is a live demo: [http://my-netdata.io](http://my-netdata.io)*
 
 > *New to netdata? Here is a live demo: [http://my-netdata.io](http://my-netdata.io)*
 

-**netdata** is an application for **real-time performance and health monitoring**. It provides **unparalleled insights, in real-time**, of everything happening on the system it runs (including key applications such as web and database servers), using **modern interactive web dashboards**.
+**netdata** is a system for **distributed real-time performance and health monitoring**.
+It provides **unparalleled insights, in real-time**, of everything happening on the
+system it runs (including applications such as web and database servers), using
+**modern interactive web dashboards**.
+
+_netdata is **fast** and **efficient**, designed to permanently run on all systems
+(**physical** & **virtual** servers, **containers**, **IoT** devices), without
+disrupting their core function._
+
+netdata runs on **Linux**, **FreeBSD**, and **MacOS**.

 
 

-_netdata is **very fast** and **super efficient**, designed to permanently run on all systems (**physical** & **virtual** servers, **containers**, **IoT** devices), without disrupting their core function._
+[![Twitter Follow](https://img.shields.io/twitter/follow/linuxnetdata.svg?style=social&label=New%20-%20stay%20in%20touch%20-%20follow%20netdata%20on%20twitter)](https://twitter.com/linuxnetdata)

 
 ---
 
 ## User base
 
 *Since May 16th 2016 (the date the [global public netdata registry](https://github.com/firehol/netdata/wiki/mynetdata-menu-item) was released):*<br/>
 
 ---
 
 ## User base
 
 *Since May 16th 2016 (the date the [global public netdata registry](https://github.com/firehol/netdata/wiki/mynetdata-menu-item) was released):*<br/>

-[![User Base](https://registry.my-netdata.io/api/v1/badge.svg?chart=netdata.registry_entries&dimensions=persons&label=user%20base&units=null&value_color=blue&precision=0&v42)](https://registry.my-netdata.io/#menu_netdata_submenu_registry) [![Monitored Servers](https://registry.my-netdata.io/api/v1/badge.svg?chart=netdata.registry_entries&dimensions=machines&label=servers%20monitored&units=null&value_color=orange&precision=0&v42)](https://registry.my-netdata.io/#menu_netdata_submenu_registry) [![Sessions Served](https://registry.my-netdata.io/api/v1/badge.svg?chart=netdata.registry_sessions&label=sessions%20served&units=null&value_color=yellowgreen&precision=0&v42)](https://registry.my-netdata.io/#menu_netdata_submenu_registry)
+[![User Base](https://registry.my-netdata.io/api/v1/badge.svg?chart=netdata.registry_entries&dimensions=persons&label=user%20base&units=null&value_color=blue&precision=0&v42)](https://registry.my-netdata.io/#menu_netdata_submenu_registry) [![Monitored Servers](https://registry.my-netdata.io/api/v1/badge.svg?chart=netdata.registry_entries&dimensions=machines&label=servers%20monitored&units=null&value_color=orange&precision=0&v42)](https://registry.my-netdata.io/#menu_netdata_submenu_registry) [![Sessions Served](https://registry.my-netdata.io/api/v1/badge.svg?chart=netdata.registry_sessions&label=sessions%20served&units=null&value_color=yellowgreen&precision=0&v42)](https://registry.my-netdata.io/#menu_netdata_submenu_registry) [![Docker Pulls](https://img.shields.io/docker/pulls/titpetric/netdata.svg)](https://hub.docker.com/r/titpetric/netdata/)

 
 *in the last 24 hours:*<br/>
 [![New Users Today](http://registry.my-netdata.io/api/v1/badge.svg?chart=netdata.registry_entries&dimensions=persons&after=-86400&options=unaligned&group=incremental-sum&label=new%20users%20today&units=null&value_color=blue&precision=0&v42)](https://registry.my-netdata.io/#menu_netdata_submenu_registry) [![New Machines Today](https://registry.my-netdata.io/api/v1/badge.svg?chart=netdata.registry_entries&dimensions=machines&group=incremental-sum&after=-86400&options=unaligned&label=servers%20added%20today&units=null&value_color=orange&precision=0&v42)](https://registry.my-netdata.io/#menu_netdata_submenu_registry) [![Sessions Today](https://registry.my-netdata.io/api/v1/badge.svg?chart=netdata.registry_sessions&after=-86400&group=incremental-sum&options=unaligned&label=sessions%20served%20today&units=null&value_color=yellowgreen&precision=0&v42)](https://registry.my-netdata.io/#menu_netdata_submenu_registry)
 
 *in the last 24 hours:*<br/>
 [![New Users Today](http://registry.my-netdata.io/api/v1/badge.svg?chart=netdata.registry_entries&dimensions=persons&after=-86400&options=unaligned&group=incremental-sum&label=new%20users%20today&units=null&value_color=blue&precision=0&v42)](https://registry.my-netdata.io/#menu_netdata_submenu_registry) [![New Machines Today](https://registry.my-netdata.io/api/v1/badge.svg?chart=netdata.registry_entries&dimensions=machines&group=incremental-sum&after=-86400&options=unaligned&label=servers%20added%20today&units=null&value_color=orange&precision=0&v42)](https://registry.my-netdata.io/#menu_netdata_submenu_registry) [![Sessions Today](https://registry.my-netdata.io/api/v1/badge.svg?chart=netdata.registry_sessions&after=-86400&group=incremental-sum&options=unaligned&label=sessions%20served%20today&units=null&value_color=yellowgreen&precision=0&v42)](https://registry.my-netdata.io/#menu_netdata_submenu_registry)


@@ -21,16 +29,20 @@ _netdata is **very fast** and **super efficient**, designed to permanently run o
 ## News
 
 <p align="center">
 ## News
 
 <p align="center">

-<a href="https://octoverse.github.com/" target="_blank"><img src="https://cloud.githubusercontent.com/assets/2662304/18795170/ec321f32-81cb-11e6-92a8-d03492f0b00d.png"/></a>
+Netdata is featured at <b><a href="https://octoverse.github.com/" target="_blank">GitHub's State Of The Octoverse 2016</a></b><br/>
+<a href="https://octoverse.github.com/" target="_blank"><img src="https://cloud.githubusercontent.com/assets/2662304/21743260/23ebe62c-d507-11e6-80c0-76b95f53e464.png"/></a>

 </p>
 
 </p>
 

-`Oct 4th, 2016` - **[netdata v1.4.0 released!](https://github.com/firehol/netdata/releases)**
+`Jan 22nd, 2017` - **[netdata v1.5.0 released!](https://github.com/firehol/netdata/releases)**

 
 

- - the **fastest** netdata ever (with a better look too)!
- - improved **IoT** and **containers** support!
- - **alarms** improved in almost every way!
- - new plugins:  softnet netdev, extended TCP metrics, UDPLite, NFS v2, v3 client (server was there already), NFS v4 server & client, APCUPSd, RetroShare
- - improved plugins: mysql, cgroups, hddtemp, sensors, phpfm, tc (QoS)
+ - netdata now runs on **FreeBSD** and **MacOS**
+ - netdata now supports **Graphite**, **OpenTSDB**, **Prometheus** and compatible backends
+ - netdata now monitors **systemd Services**
+ - new plugins: fping, postgres, varnish, elasticsearch, haproxy, freeradius, mdstat, ISC dhcpd, fail2ban, openvpn, NUMA memory, CPU Idle States, gunicorn, ECC memory errors, IPC semaphores, uptime
+ - improved plugins: netfilter conntrack, mysql/mariadb, ipfs, cpufreq, hddtemp, sensors, nginx, nginx_log, phpfpm, redis, dovecot, containers and cgroups, disk space, apps.plugin, tc (QoS) and almost all internal plugins (memory, IPv4 and IPv6, network interfaces, QoS, etc)
+ - dozens of new and improved alarms (including performance monitoring alarms for mysql)
+ - new alarm notifications: messagebird.com, pagerduty.com, pushbullet.com, twilio.com, hipchat, kafka
+ - dozens more improvements and performance optimizations

 
 ---
 
 
 ---
 


@@ -40,17 +52,55 @@ _netdata is **very fast** and **super efficient**, designed to permanently run o
 <img src="https://cloud.githubusercontent.com/assets/2662304/19168687/f6a567be-8c19-11e6-8561-ce8d589e8346.gif"/>
 </p>
 
 <img src="https://cloud.githubusercontent.com/assets/2662304/19168687/f6a567be-8c19-11e6-8561-ce8d589e8346.gif"/>
 </p>
 

- - **Stunning interactive bootstrap dashboards**<br/>2 themes are provided: dark, light
- - **Blazingly fast visualization**<br/>responds to all queries in less than 0.5 ms per metric, even on low-end hardware
- - **Highly efficient data collection**<br/>collects thousands of metrics per server per second, with just 1% CPU utilization of a single core on modern hardware
- - **Sophisticated alarming**<br/>supports dynamic thresholds, hysteresis, templating, multiple notification methods
- - **Extensible**<br/>you can monitor anything you can get a metric for, using its Plugin API (anything can be a netdata plugin, BASH, python, perl, node.js, java, Go, etc)
- - **Zero configuration**<br/>auto-detects everything, out of the box it can collect up to 5000 metrics per server
- - **Zero dependencies**<br/>it is even its own web server, for its static web files and its web API
- - **Zero maintenance**<br/>you just run it, it does the rest
- - **Embeddable**<br/>it can run anywhere a Linux kernel runs (even IoT) and its charts can be embedded on your web pages too
- - **Custom dashboards**<br/>that can be built using simple HTML (no javascript necessary)
- - **scales to infinity**<br/>requiring minimal central resources!
+ - **Stunning interactive bootstrap dashboards**<br/>
+   mouse and touch friendly, in 2 themes: dark, light
+   
+ - **Amazingly fast**<br/>
+   responds to all queries in less than 0.5 ms per metric,
+   even on low-end hardware
+   
+ - **Highly efficient**<br/>
+   collects thousands of metrics per server per second,
+   with just 1% CPU utilization of a single core, a few MB of RAM and no disk I/O at all
+   
+ - **Sophisticated alarming**<br/>
+   hundreds of alarms, **out of the box**!<br/>
+   supports dynamic thresholds, hysteresis, alarm templates,
+   multiple role-based notification methods (such as email, slack.com,
+   pushover.net, pushbullet.com, telegram.org, twilio.com, messagebird.com)
+   
+ - **Extensible**<br/>
+   you can monitor anything you can get a metric for,
+   using its Plugin API (anything can be a netdata plugin,
+   BASH, python, perl, node.js, java, Go, ruby, etc)
+   
+ - **Embeddable**<br/>
+   it can run anywhere a Linux kernel runs (even IoT)
+   and its charts can be embedded on your web pages too
+   
+ - **Customizable**<br/>
+   custom dashboards can be built using simple HTML (no javascript necessary)
+   
+ - **Zero configuration**<br/>
+   auto-detects everything, it can collect up to 5000 metrics
+   per server out of the box
+   
+ - **Zero dependencies**<br/>
+   it is even its own web server, for its static web files and its web API
+   
+ - **Zero maintenance**<br/>
+   you just run it, it does the rest
+   
+ - **scales to infinity**<br/>
+   requiring minimal central resources
+   
+ - **several operating modes**<br/>
+   autonomous host monitoring, headless data collector, forwarding proxy, store and forward proxy, central multi-host monitoring, in all possible configurations.
+   Each node may have different metrics retention policy and run with or without health monitoring.
+   
+ - **time-series back-ends supported**<br/>
+   can archive its metrics on `graphite`, `opentsdb`, `prometheus`, json document DBs, in the same or lower detail
+   (lower: to prevent it from congesting these servers due to the amount of data collected)

 
 ![netdata](https://cloud.githubusercontent.com/assets/2662304/14092712/93b039ea-f551-11e5-822c-beadbf2b2a2e.gif)
 
 
 ![netdata](https://cloud.githubusercontent.com/assets/2662304/14092712/93b039ea-f551-11e5-822c-beadbf2b2a2e.gif)
 


@@ -58,77 +108,169 @@ _netdata is **very fast** and **super efficient**, designed to permanently run o
 
 ## What does it monitor?
 
 
 ## What does it monitor?
 

-This is what it currently monitors (most with zero configuration):
+netdata collects several thousands of metrics per device.
+All these metrics are collected and visualized in real-time.
+
+> _Almost all metrics are auto-detected, without any configuration._
+
+This is a list of what it currently monitors:

 
 

-- **CPU usage, interrupts, softirqs and frequency** (total and per core)
+- **CPU**<br/>
+  usage, interrupts, softirqs, frequency, total and per core, CPU states

 
 

-- **RAM, swap and kernel memory usage** (including KSM and kernel memory deduper)
+- **Memory**<br/>
+  RAM, swap and kernel memory usage, KSM (Kernel Samepage Merging), NUMA

 
 

-- **Disks** (per disk: I/O, operations, backlog, utilization, space, etc)
+- **Disks**<br/>
+  per disk: I/O, operations, backlog, utilization, space, software RAID (md)

 
    ![sda](https://cloud.githubusercontent.com/assets/2662304/14093195/c882bbf4-f554-11e5-8863-1788d643d2c0.gif)
 
 
    ![sda](https://cloud.githubusercontent.com/assets/2662304/14093195/c882bbf4-f554-11e5-8863-1788d643d2c0.gif)
 

-- **Network interfaces** (per interface: bandwidth, packets, errors, drops, etc)
+- **Network interfaces**<br/>
+  per interface: bandwidth, packets, errors, drops

 
    ![dsl0](https://cloud.githubusercontent.com/assets/2662304/14093128/4d566494-f554-11e5-8ee4-5392e0ac51f0.gif)
 
 
    ![dsl0](https://cloud.githubusercontent.com/assets/2662304/14093128/4d566494-f554-11e5-8ee4-5392e0ac51f0.gif)
 

-- **IPv4 networking** (bandwidth, packets, errors, fragments, tcp: connections, packets, errors, handshake, udp: packets, errors, broadcast: bandwidth, packets, multicast: bandwidth, packets)
+- **IPv4 networking**<br/>
+  bandwidth, packets, errors, fragments,
+  tcp: connections, packets, errors, handshake,
+  udp: packets, errors,
+  broadcast: bandwidth, packets,
+  multicast: bandwidth, packets

 
 

-- **IPv6 networking** (bandwidth, packets, errors, fragments, ECT, udp: packets, errors, udplite: packets, errors, broadcast: bandwidth, multicast: bandwidth, packets, icmp: messages, errors, echos, router, neighbor, MLDv2, group membership, break down by type)
+- **IPv6 networking**<br/>
+  bandwidth, packets, errors, fragments, ECT,
+  udp: packets, errors,
+  udplite: packets, errors,
+  broadcast: bandwidth,
+  multicast: bandwidth, packets,
+  icmp: messages, errors, echos, router, neighbor, MLDv2, group membership,
+  break down by type

 
 

-- **netfilter / iptables Linux firewall** (connections, connection tracker events, errors, etc)
+- **Interprocess Communication - IPC**<br/>
+  such as semaphores and semaphores arrays

 
 

-- **Linux DDoS protection** (SYNPROXY metrics)
+- **netfilter / iptables Linux firewall**<br/>
+  connections, connection tracker events, errors

 
 

-- **Processes** (running, blocked, forks, active, etc)
+- **Linux DDoS protection**<br/>
+  SYNPROXY metrics

 
 

-- **Entropy** (random numbers pool, using in cryptography)
+- **fping** latencies</br>
+  for any number of hosts, showing latency, packets and packet loss

 
 

-- **NFS file servers and clients**, v2, v3, v4 (I/O, cache, read ahead, RPC calls)
+   ![image](https://cloud.githubusercontent.com/assets/2662304/20464811/9517d2b4-af57-11e6-8361-f6cc57541cd7.png)

 
 

-- **Network QoS** (yes, the only tool that visualizes network `tc` classes in realtime)
+
+- **Processes**<br/>
+  running, blocked, forks, active
+
+- **Entropy**<br/>
+  random numbers pool, using in cryptography
+
+- **NFS file servers and clients**<br/>
+  NFS v2, v3, v4: I/O, cache, read ahead, RPC calls
+
+- **Network QoS**<br/>
+  the only tool that visualizes network `tc` classes in realtime

 
    ![qos-tc-classes](https://cloud.githubusercontent.com/assets/2662304/14093004/68966020-f553-11e5-98fe-ffee2086fafd.gif)
 
 
    ![qos-tc-classes](https://cloud.githubusercontent.com/assets/2662304/14093004/68966020-f553-11e5-98fe-ffee2086fafd.gif)
 

-- **Linux Control Groups** (containers), systemd, lxc, docker, etc
+- **Linux Control Groups**<br/>
+  containers: systemd, lxc, docker

 
 

-- **Applications**, by grouping the process tree (CPU, memory, disk reads, disk writes, swap, threads, pipes, sockets, etc)
+- **Applications**<br/>
+  by grouping the process tree and reporting CPU, memory, disk reads,
+  disk writes, swap, threads, pipes, sockets - per group

 
    ![apps](https://cloud.githubusercontent.com/assets/2662304/14093565/67c4002c-f557-11e5-86bd-0154f5135def.gif)
 
 
    ![apps](https://cloud.githubusercontent.com/assets/2662304/14093565/67c4002c-f557-11e5-86bd-0154f5135def.gif)
 

-- **Users and User Groups resource usage**, by summarizing the process tree per user and group (CPU, memory, disk reads, disk writes, swap, threads, pipes, sockets, etc)
+- **Users and User Groups resource usage**<br/>
+  by summarizing the process tree per user and group,
+  reporting: CPU, memory, disk reads, disk writes, swap, threads, pipes, sockets
+
+- **Apache and lighttpd web servers**<br/>
+   `mod-status` (v2.2, v2.4) and cache log statistics, for multiple servers
+
+- **Nginx web servers**<br/>
+  `stub-status`, for multiple servers
+
+- **Tomcat**<br/>
+  accesses, threads, free memory, volume
+
+- **web server log files**<br/>
+  extracting in real-time, web server performance metrics and applying several health checks
+
+- **mySQL databases**<br/>
+  multiple servers, each showing: bandwidth, queries/s, handlers, locks, issues,
+  tmp operations, connections, binlog metrics, threads, innodb metrics, and more
+
+- **Postgres databases**<br/>
+  multiple servers, each showing: per database statistics (connections, tuples
+  read - written - returned, transactions, locks), backend processes, indexes,
+  tables, write ahead, background writer and more
+
+- **Redis databases**<br/>
+  multiple servers, each showing: operations, hit rate, memory, keys, clients, slaves
+
+- **mongodb**<br/>
+  operations, clients, transactions, cursors, connections, asserts, locks, etc
+
+- **memcached databases**<br/>
+  multiple servers, each showing: bandwidth, connections, items
+
+- **elasticsearch**<br/>
+  search and index performance, latency, timings, cluster statistics, threads statistics, etc
+
+- **ISC Bind name servers**<br/>
+  multiple servers, each showing: clients, requests, queries, updates, failures and several per view metrics

 
 

-- **Apache web servers** mod-status (v2.2, v2.4) and cache log statistics (multiple servers - compatible with lighttpd too)
+- **NSD name servers**<br/>
+  queries, zones, protocols, query types, transfers, etc.

 
 

-- **Nginx web servers** stub-status (multiple servers)
+- **Postfix email servers**<br/>
+  message queue (entries, size)

 
 

-- **mySQL databases** (multiple servers, each showing: bandwidth, queries/s, handlers, locks, issues, tmp operations, connections, binlog metrics, threads, innodb metrics, etc)
+- **exim email servers**<br/>
+  message queue (emails queued)

 
 

-- **Redis databases** (multiple servers, each showing: operations, hit rate, memory, keys, clients, slaves)
+- **Dovecot** POP3/IMAP servers<br/>

 
 

-- **memcached databases** (multiple servers, each showing: bandwidth, connections, items, etc)
+- **ISC dhcpd**<br/>
+  pools utilization, leases, etc.

 
 

-- **ISC Bind name servers** (multiple servers, each showing: clients, requests, queries, updates, failures and several per view metrics)
+- **IPFS**<br/>
+  bandwidth, peers

 
 

-- **Postfix email servers** message queue (entries, size)
+- **Squid proxy servers**<br/>
+  multiple servers, each showing: clients bandwidth and requests, servers bandwidth and requests

 
 

-- **exim email servers** message queue (emails queued)
+- **HAproxy**<br/>
+  bandwidth, sessions, backends, etc

 
 

-- **IPFS** (Bandwidth, Peers)
+- **varnish**<br/>
+  threads, sessions, hits, objects, backends, etc

 
 

-- **Squid proxy servers** (multiple servers, each showing: clients bandwidth and requests, servers bandwidth and requests)
+- **OpenVPN**<br/>
+  status per tunnel

 
 

-- **Hardware sensors** (temperature, voltage, fans, power, humidity, etc)
+- **Hardware sensors**<br/>
+  `lm_sensors` and `IPMI`: temperature, voltage, fans, power, humidity

 
 

-- **NUT and APC UPSes** (load, charge, battery voltage, temperature, utility metrics, output metrics)
+- **NUT and APC UPSes**<br/>
+  load, charge, battery voltage, temperature, utility metrics, output metrics

 
 

-- **Tomcat** (accesses, threads, free memory, volume)
+- **PHP-FPM**<br/>
+  multiple instances, each reporting connections, requests, performance

 
 

-- **PHP-FPM** (multiple instances, each reporting connections, requests, performance)
+- **hddtemp**<br/>
+  disk temperatures

 
 

-- **hddtemp** (disk temperatures)
+- **smartd**<br/>
+  disk S.M.A.R.T. values

 
 

-- **SNMP devices** can be monitored too (although you will need to configure these)
+- **SNMP devices**<br/>
+  can be monitored too (although you will need to configure these)

 
 And you can extend it, by writing plugins that collect data from any source, using any computer language.
 
 
 And you can extend it, by writing plugins that collect data from any source, using any computer language.
 


@@ -146,7 +288,7 @@ It should run on **any Linux** system (including IoT). It has been tested on:
 - Debian
 - Fedora
 - Gentoo
 - Debian
 - Fedora
 - Gentoo

-- OpenSuse
+- openSUSE

 - PLD Linux
 - RedHat Enterprise Linux
 - SUSE
 - PLD Linux
 - RedHat Enterprise Linux
 - SUSE