-netdata
-=======
+# netdata [](https://travis-ci.org/firehol/netdata) [](https://scan.coverity.com/projects/firehol-netdata) [](https://www.codacy.com/app/netdata/netdata?utm_source=github.com&utm_medium=referral&utm_content=firehol/netdata&utm_campaign=Badge_Grade) [](https://codeclimate.com/github/firehol/netdata)
+> *New to netdata? Here is a live demo: [http://my-netdata.io](http://my-netdata.io)*
+
+**netdata** is a system for **distributed real-time performance and health monitoring**.
+It provides **unparalleled insights, in real-time**, of everything happening on the
+system it runs (including applications such as web and database servers), using
+**modern interactive web dashboards**.
+
+_netdata is **fast** and **efficient**, designed to permanently run on all systems
+(**physical** & **virtual** servers, **containers**, **IoT** devices), without
+disrupting their core function._
+
+netdata runs on **Linux**, **FreeBSD**, and **MacOS**.
+
+[](https://twitter.com/linuxnetdata)
+
+---
+
+## User base
+
+*Since May 16th 2016 (the date the [global public netdata registry](https://github.com/firehol/netdata/wiki/mynetdata-menu-item) was released):*<br/>
+[](https://registry.my-netdata.io/#menu_netdata_submenu_registry) [](https://registry.my-netdata.io/#menu_netdata_submenu_registry) [](https://registry.my-netdata.io/#menu_netdata_submenu_registry) [](https://hub.docker.com/r/titpetric/netdata/)
+
+*in the last 24 hours:*<br/>
+[](https://registry.my-netdata.io/#menu_netdata_submenu_registry) [](https://registry.my-netdata.io/#menu_netdata_submenu_registry) [](https://registry.my-netdata.io/#menu_netdata_submenu_registry)
+
+---
+
+## News
+
+<p align="center">
+Netdata is featured at <b><a href="https://octoverse.github.com/" target="_blank">GitHub's State Of The Octoverse 2016</a></b><br/>
+<a href="https://octoverse.github.com/" target="_blank"><img src="https://cloud.githubusercontent.com/assets/2662304/21743260/23ebe62c-d507-11e6-80c0-76b95f53e464.png"/></a>
+</p>
+
+`Mar 20th, 2017` - **[netdata v1.6.0 released!](https://github.com/firehol/netdata/releases)**
+
+ - central netdata is here! headless collectors, proxies, streaming of metrics, etc.
+ - [monitoring ephemeral nodes (auto-scaled VMs)](https://github.com/firehol/netdata/wiki/monitoring-ephemeral-nodes)
+ - [monitoring ephemeral containers and VM guests](https://github.com/firehol/netdata/wiki/monitoring-ephemeral-containers)
+ - [monitoring web servers](https://github.com/firehol/netdata/wiki/The-spectacles-of-a-web-server-log-file)
+ - apps.plugin ported for FreeBSD
+ - [monitoring IPMI](https://github.com/firehol/netdata/wiki/monitoring-IPMI)
+ - dozens of new and improved plugins
+ - dozens of new and improved alarms
+ - dozens more improvements and performance optimizations
+
+---
+
+## Features
+
+<p align="center">
+<img src="https://cloud.githubusercontent.com/assets/2662304/19168687/f6a567be-8c19-11e6-8561-ce8d589e8346.gif"/>
+</p>
+
+ - **Stunning interactive bootstrap dashboards**<br/>
+ mouse and touch friendly, in 2 themes: dark, light
+
+ - **Amazingly fast**<br/>
+ responds to all queries in less than 0.5 ms per metric,
+ even on low-end hardware
+
+ - **Highly efficient**<br/>
+ collects thousands of metrics per server per second,
+ with just 1% CPU utilization of a single core, a few MB of RAM and no disk I/O at all
+
+ - **Sophisticated alarming**<br/>
+ hundreds of alarms, **out of the box**!<br/>
+ supports dynamic thresholds, hysteresis, alarm templates,
+ multiple role-based notification methods (such as email, slack.com,
+ pushover.net, pushbullet.com, telegram.org, twilio.com, messagebird.com)
+
+ - **Extensible**<br/>
+ you can monitor anything you can get a metric for,
+ using its Plugin API (anything can be a netdata plugin,
+ BASH, python, perl, node.js, java, Go, ruby, etc)
+
+ - **Embeddable**<br/>
+ it can run anywhere a Linux kernel runs (even IoT)
+ and its charts can be embedded on your web pages too
+
+ - **Customizable**<br/>
+ custom dashboards can be built using simple HTML (no javascript necessary)
+
+ - **Zero configuration**<br/>
+ auto-detects everything, it can collect up to 5000 metrics
+ per server out of the box
+
+ - **Zero dependencies**<br/>
+ it is even its own web server, for its static web files and its web API
+
+ - **Zero maintenance**<br/>
+ you just run it, it does the rest
+
+ - **scales to infinity**<br/>
+ requiring minimal central resources
+
+ - **several operating modes**<br/>
+ autonomous host monitoring, headless data collector, forwarding proxy, store and forward proxy, central multi-host monitoring, in all possible configurations.
+ Each node may have different metrics retention policy and run with or without health monitoring.
+
+ - **time-series back-ends supported**<br/>
+ can archive its metrics on `graphite`, `opentsdb`, `prometheus`, json document DBs, in the same or lower detail
+ (lower: to prevent it from congesting these servers due to the amount of data collected)
-[Live Demo Site](http://netdata.firehol.org)
+
+---
-### Realtime time data collection and charts!
+## What does it monitor?
-**Netdata** is a daemon that collects data in realtime (per second) and presents a web site to view and analyze them.
-The presentation is full of charts that precisely render all system values, in realtime.
+netdata collects several thousands of metrics per device.
+All these metrics are collected and visualized in real-time.
-It has been designed to be installed **on every system**, without desrupting it:
+> _Almost all metrics are auto-detected, without any configuration._
-1. It will just use some spare CPU cycles.
+This is a list of what it currently monitors:
+
+- **CPU**<br/>
+ usage, interrupts, softirqs, frequency, total and per core, CPU states
+
+- **Memory**<br/>
+ RAM, swap and kernel memory usage, KSM (Kernel Samepage Merging), NUMA
+
+- **Disks**<br/>
+ per disk: I/O, operations, backlog, utilization, space, software RAID (md)
+
+ 
+
+- **Network interfaces**<br/>
+ per interface: bandwidth, packets, errors, drops
+
+ 
+
+- **IPv4 networking**<br/>
+ bandwidth, packets, errors, fragments,
+ tcp: connections, packets, errors, handshake,
+ udp: packets, errors,
+ broadcast: bandwidth, packets,
+ multicast: bandwidth, packets
+
+- **IPv6 networking**<br/>
+ bandwidth, packets, errors, fragments, ECT,
+ udp: packets, errors,
+ udplite: packets, errors,
+ broadcast: bandwidth,
+ multicast: bandwidth, packets,
+ icmp: messages, errors, echos, router, neighbor, MLDv2, group membership,
+ break down by type
- Although it is very light weight, on slow processors you can futher control its CPU consumption by lowering its data collection frequency.
- By default it is running with the lowest possible linux priority.
+- **Interprocess Communication - IPC**<br/>
+ such as semaphores and semaphores arrays
-2. It will use the memory you want it have.
+- **netfilter / iptables Linux firewall**<br/>
+ connections, connection tracker events, errors
- Although we have made the most to make its memory footprint the smallest possible,
- you can further control the memory it will use, by sizing its round robin memory database.
+- **Linux DDoS protection**<br/>
+ SYNPROXY metrics
-3. It does not use disk I/O.
+- **fping** latencies</br>
+ for any number of hosts, showing latency, packets and packet loss
- All its round robin database is in memory.
- It is only saved on disk and loaded back when netdata restarts.
- You can also disable the access log of its embedded web server, so that netdata will not use any I/O at all while running.
+ 
-You can use it to monitor all your applications, servers, linux PCs or linux embedded devices.
+- **Processes**<br/>
+ running, blocked, forks, active
-Out of the box, it comes with plugins for data collection about system information and popular applications.
+- **Entropy**<br/>
+ random numbers pool, using in cryptography
+- **NFS file servers and clients**<br/>
+ NFS v2, v3, v4: I/O, cache, read ahead, RPC calls
-# Features
+- **Network QoS**<br/>
+ the only tool that visualizes network `tc` classes in realtime
-- **highly optimized C code**
+ 
- It only needs a few milliseconds per second to collect all the data.
- It will nicelly run even on a raspberry pi with just one cpu core, or any other embedded system.
+- **Linux Control Groups**<br/>
+ containers: systemd, lxc, docker
-- **extremely lightweight**
+- **Applications**<br/>
+ by grouping the process tree and reporting CPU, memory, disk reads,
+ disk writes, swap, threads, pipes, sockets - per group
- It only needs a few megabytes of memory to store its round robin database.
+ 
- Although `netdata` does all its calculation using `long double` (128 bit) arithmetics, it stores all values using a **custom-made 32-bit number**. This custom-made number can store in 29 bits values from -167772150000000.0 to 167772150000000.0 with a precision of 0.00001 (yes, it is a floating point number, meaning that higher integer values have less decimal precision) and 3 bits for flags. This provides an extremely optimized memory footprint with just 0.0001% max accuracy loss (run: `./netdata --unittest` to see it in action).
+- **Users and User Groups resource usage**<br/>
+ by summarizing the process tree per user and group,
+ reporting: CPU, memory, disk reads, disk writes, swap, threads, pipes, sockets
- If your linux box has KSM enabled, netdata will give it all its round robbin database, to lower its memory requirements even further.
+- **Apache and lighttpd web servers**<br/>
+ `mod-status` (v2.2, v2.4) and cache log statistics, for multiple servers
-- **per second data collection**
+- **Nginx web servers**<br/>
+ `stub-status`, for multiple servers
- Every chart, every value, is updated every second. Of course, you can control collection period per plugin.
+- **Tomcat**<br/>
+ accesses, threads, free memory, volume
- **netdata** can perform several calculations on each value (dimension) collected:
+- **web server log files**<br/>
+ extracting in real-time, web server performance metrics and applying several health checks
- - **absolute**, stores the collected value, as collected (this is used, for example for the number of processes running, the number of connections open, the amount of RAM used, etc)
+- **mySQL databases**<br/>
+ multiple servers, each showing: bandwidth, queries/s, handlers, locks, issues,
+ tmp operations, connections, binlog metrics, threads, innodb metrics, and more
- - **incremental**, stores the difference of the collected value to the last collected value (this is used, for example, for the bandwidth of interfaces, disk I/O, i.e. for counters that always get incremented) - **netdata** automatically interpolates these values to second boundary, using nanosecond calculations so that small delays at the data collection layer will not affect the quality of the result - also, **netdata** detects arithmetic overflows and presents them properly at the charts.
+- **Postgres databases**<br/>
+ multiple servers, each showing: per database statistics (connections, tuples
+ read - written - returned, transactions, locks), backend processes, indexes,
+ tables, write ahead, background writer and more
- - **percentage of absolute row**, stores the percentage of the collected value, over the sum of all dimensions of the chart.
+- **Redis databases**<br/>
+ multiple servers, each showing: operations, hit rate, memory, keys, clients, slaves
- - **percentage of incremental row**, stores the percentage of this collected value, over the sum of the the **incremental** differences of all dimensions of the chart (this is used, for example, for system CPU utilization).
+- **mongodb**<br/>
+ operations, clients, transactions, cursors, connections, asserts, locks, etc
-- **visualizes QoS classes automatically**
+- **memcached databases**<br/>
+ multiple servers, each showing: bandwidth, connections, items
- If you also use FireQOS for QoS, it collects class names automatically.
+- **elasticsearch**<br/>
+ search and index performance, latency, timings, cluster statistics, threads statistics, etc
-- **appealing web site**
+- **ISC Bind name servers**<br/>
+ multiple servers, each showing: clients, requests, queries, updates, failures and several per view metrics
- The web site uses bootstrap and the excellent [dygraphs](http://dygraphs.com), for a very appealing and responsive result.
- It works even on mobile devices and adapts to screen size changes and rotation (responsive design).
+- **NSD name servers**<br/>
+ queries, zones, protocols, query types, transfers, etc.
-- **web charts do respect your browser resources**
+- **Postfix email servers**<br/>
+ message queue (entries, size)
- The charts adapt to show only as many points are required to have a clear view.
- Also, the javascript code respects your browser resources (stops refreshing when the window looses focus, when something is selected, when charts are not in the visible viewport, etc).
+- **exim email servers**<br/>
+ message queue (emails queued)
-- **highly configurable**
+- **Dovecot** POP3/IMAP servers<br/>
- All charts and all features can be enabled or disabled.
- The program generates its configuration file based on the resources available on the system it runs, for you to edit.
+- **ISC dhcpd**<br/>
+ pools utilization, leases, etc.
-- It reads and renders charts for all these:
- - `/proc/net/dev` (all netwrok interfaces for all their values)
- - `/proc/diskstats` (all disks for all their values)
- - `/proc/net/snmp` (total IPv4, TCP and UDP usage)
- - `/proc/net/netstat` (more IPv4 usage)
- - `/proc/net/stat/nf_conntrack` (connection tracking performance)
- - `/proc/net/ip_vs/stats` (IPVS connection statistics)
- - `/proc/stat` (CPU utilization)
- - `/proc/meminfo` (memory information)
- - `/proc/vmstat` (system performance)
- - `/proc/net/rpc/nfsd` (NFS server statistics for both v3 and v4 NFS)
- - `/proc/interrupts` (total and per core hardware interrupts)
- - `/proc/softirqs` (total and per core software interrupts)
- - `tc` classes (QoS classes - [with FireQOS class names](http://firehol.org/tutorial/fireqos-new-user/))
+- **IPFS**<br/>
+ bandwidth, peers
-- It supports **plugins** for collecting information from other sources!
+- **Squid proxy servers**<br/>
+ multiple servers, each showing: clients bandwidth and requests, servers bandwidth and requests
- Plugins can be written in any computer language (pipe / stdout communication for data collection).
+- **HAproxy**<br/>
+ bandwidth, sessions, backends, etc
- It ships with 2 plugins: `apps.plugin` and `charts.d.plugin`:
+- **varnish**<br/>
+ threads, sessions, hits, objects, backends, etc
- - `apps.plugin` is a plugin that attempts to collect statistics per process. It groups the entire process tree based on your settings (for example, mplayer, kodi, vlc are all considered `media`) and for each group it attempts to find CPU usage, memory usages, physical and logical disk read and writes, number of processes, number of threads, number of open files, number of open sockets, number of open pipes, minor and major page faults (major = swapping), etc. 15 stackable (per group) charts in total.
+- **OpenVPN**<br/>
+ status per tunnel
- - `charts.d.plugin` provides a simple way to script data collection in BASH. It includes example plugins that collect values from:
+- **Hardware sensors**<br/>
+ `lm_sensors` and `IPMI`: temperature, voltage, fans, power, humidity
- - `nut` (UPS load, frequency, voltage, etc, for multiple UPSes)
- - `sensors` (temperature, voltage, current, power, humidity, fans rotation sensors)
- - `cpufreq` (current CPU clock frequency, for all CPUs)
- - `postfix` (e-mail queue size)
- - `squid` (web proxy statistics)
- - `mysql` (mysql global statistics)
- - `opensips` (opensips statistics)
+- **NUT and APC UPSes**<br/>
+ load, charge, battery voltage, temperature, utility metrics, output metrics
- Of course, you can write your own using BASH scripting.
+- **PHP-FPM**<br/>
+ multiple instances, each reporting connections, requests, performance
-- netdata is a web server, supporting gzip compression
+- **hddtemp**<br/>
+ disk temperatures
- It serves its own static files and dynamic files for rendering the site.
- It does not support authentication or SSL - limit its access using your firewall, or put it behind an authentication proxy.
- It does not allow ` .. ` in the files requested (so it can only serve files stored in the web directory `/usr/share/netdata/web`).
+- **smartd**<br/>
+ disk S.M.A.R.T. values
+- **SNMP devices**<br/>
+ can be monitored too (although you will need to configure these)
-# How it works
+And you can extend it, by writing plugins that collect data from any source, using any computer language.
-1. You run a daemon on your linux: netdata.
- This deamon is written in C and is extremely lightweight.
+---
- netdata:
+## Installation
- - Spawns threads to collect all the data for all sources
- - Keeps track of the collected values in memory (no disk I/O at all)
- - Generates JSON and JSONP HTTP responses containing all the data needed for the web graphs
- - Is a standalone web server.
+Use our **[automatic installer](https://github.com/firehol/netdata/wiki/Installation)** to build and install it on your system.
- For example, you can access JSON data by using:
+It should run on **any Linux** system (including IoT). It has been tested on:
- ```
-
- http://netdata.firehol.org/api/v1/data?chart=net.eth0&after=-300&before=0&points=120&group=average&format=json
+- Alpine
+- Arch Linux
+- CentOS
+- Debian
+- Fedora
+- Gentoo
+- openSUSE
+- PLD Linux
+- RedHat Enterprise Linux
+- SUSE
+- Ubuntu
- ```
+---
- The above will give you the last 300 seconds of traffic for eth0, aggregated in 120 points, grouped as averages, in json format.
+## Documentation
- Check [Netdata Swagger UI](http://netdata.firehol.org/swagger/) for more information about the API.
+Check the **[netdata wiki](https://github.com/firehol/netdata/wiki)**.
-
-2. If you need to embed a **netdata** chart on your web page, you can add a few javascript lines and a `div` for every graph you need. Check [this example](http://netdata.firehol.org/dashboard.html) (open it in a new tab and view its source to get the idea).
+## License
-3. No internet access is required. Netdata is standalone.
-
-
-# Installation
-
-## Automatic installation
-
-Before you start, make sure you have `zlib` development files installed.
-
-You also need to have a basic build environment in place. You will need packages like
-`gcc`, `autoconf`, `autogen`, `automake`, `pgk-config`, etc.
-
-To install them in debian/ubuntu, you need to run:
-
-```sh
-apt-get install zlib1g-dev gcc autoconf autogen automake pkg-config
-```
-
-Then do this to install and run netdata:
-
-```sh
-
-# download it
-git clone https://github.com/firehol/netdata.git netdata.git
-cd netdata.git
-
-# build it
-./netdata-installer.sh
-
-```
-
-The script `netdata-installer.sh` will build netdata and install it to your system.
-
-Once the installer completes, the file `/etc/netdata/netdata.conf` will be created.
-You can edit this file to set options. To apply the changes you made, you have to restart netdata.
-
-- You can start netdata by executing it with `/usr/sbin/netdata` (the installer will also start it).
-
-- You can stop netdata by killing it with `killall netdata`.
- You can stop and start netdata at any point. Netdata saves on exit its round robbin
- database to `/var/cache/netdata` so that it will continue from where it stopped the last time.
-
-To access the web site for all graphs, go to:
-
- ```
- http://127.0.0.1:19999/
- ```
-
-You can get the running config file at any time, by accessing `http://127.0.0.1:19999/netdata.conf`.
-
-To start it at boot, just run `/usr/sbin/netdata` from your `/etc/rc.local` or equivalent.
+netdata is GPLv3+.
+It re-distributes other open-source tools and libraries. Please check its [License Statement](https://github.com/firehol/netdata/blob/master/LICENSE.md).
projects / netdata.git / blobdiff